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ABSTRACT 


Effective  decision  making  in  aeronautical  environments,  which  often  involves  high 
elements  of  risk,  is  critical  to  mission  success.  Unfortunately,  no  proven  methodology 
exists  to  train  pilots  to  make  successful  decisions.  Cockpit  decision  making  has  relied  on 
traditional  analytical  models  and  methodologies  that  underestimate  the  role  of  pilot 
experience,  expertise  and  judgment.  Naturalistic  Decision  Malang  models  (NDM) 
contend  that  decision  makers  facing  real-world  decisions  use  experience  and  judgment  to 
make  timely  decisions  without  analyzing  a  multitude  of  alternatives. 

This  thesis  analyzes  438  P-3  aviation  hazard  reports  (hazreps)  to  ascertain  which 
cognitive  strategies  from  either  the  analytical  or  naturalistic  methodology  are  more 
appropriate  for  handling  malfunction  situations.  The  author  presents  a  hybrid  model  of 
decision  making  by  P-3  pilots  based  on  the  results  of  the  analysis  and  strategies  from  both 
methodologies. 

This  thesis  recommends  that  decision  making  training  be  treated  as  a  core  activity 
of  pilots  not  only  in  flight  school,  but  after  qualification  is  complete.  Training  pilots  to 
become  experts  will  improve  situational  awareness  and  reduce  the  number  of  unfavorable 
outcomes  in  hazardous  situations. 
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L  INTRODUCTION 


A.  BACKGROUND 

Aviation  is  a  rich  domain  for  studying  decision  making.  Pilots  frequently  face 
situations  in  which  the  consequence  of  an  incorrect  decision  may  be  the  loss  of  life  or 
expensive  equipment.  These  decisions  often  are  made  in  situations  that  present 
ambiguous  information,  impose  severe  time  constraints  and  contain  exceedingly  high 
levels  of  risk.  Given  the  costs  and  high  levels  of  risk,  it  stands  to  reason  that  the  military 
would  have  conducted  thorough  studies  of  cockpit  decision  making  and  developed  a 
rigorous  approach  to  training  decision  makers.  However,  this  is  not  the  case.  In  fact, 
pilot  decision  making  skills  often  are  considered  a  result  of  such  nebulous  concepts  as 
"professionalism,"  "good  headwork"  or  merely  the  "right  stuff"  In  the  absence  of 
empirical  knowledge  regarding  the  best  way  to  train  decision  making  skills,  standard 
analytical  models  of  decision  making  simply  have  been  assumed  to  be  the  most 
appropriate  methods  (Stokes,  Kemper,  &  Kite,  1997).  For  example,  Besco  et  al.  (1994) 
asserted; 


Decision  making  in  the  cockpit  follows  traditional  views  of  decision 
making. .  .in  which  the  decision  maker  is:  1)  presented  with  a  situation  that 
requires  a  decision;  2)  the  nature  of  the  situation  is  assessed  by  the 
decision  maker;  who  3)  determines  the  availability  of  alternative  outcomes 
to  respond  to  the  situation;  and  4)  after  evaluating  the  risks  and  benefits  of 
each  alternative;  5)  selects  an  alternative  in  response  to  the  needs  of  the 
situation.  (Besco,  Maurino,  Potter,  Strauch,  Stone  &  Wiener,  1994,  p.  43) 
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Although  the  analytical  method  of  decision  making  is  clearly  valid  under  some 
conditions,  it  does  not  take  into  account  the  experience,  judgment  and  expertise  of  the 
pilot.  The  field  of  Naturalistic  Decision  Making  contends  that  decision  makers  faced  with 
real-world  decisions  use  experience  and  judgment  to  make  timely  decisions  in  actual 
settings  without  analyzing  a  multitude  of  alternatives;  indeed,  they  jfrequently  consider 
only  a  single,  workable  course  of  action.  " 

While  decision  training  is  incorporated  into  the  Navy's  Aircrew  Coordination 
Training  course,  it  is  based  solely  on  the  analytical  methodology.  To  the  author's 
knowledge,  it  does  not  recognize  many  of  the  cognitive  strategies  used  in  Naturalistic 
Decision  Making  that  could  benefit  pilots  faced  with  ambiguous,  unstructured  situations. 

B.  OBJECTIVES 

This  thesis  examines  decision  making  in  P-3  flight  stations  involving  situations 
that  meet  the  criteria  for  submission  of  a  Naval  Aviation  Safety  Program  hazard  report. 
The  thesis  objectives  are  to;  (1)  ascertain  what  types  of  decisions  are  made  in  different 
situations;  (2)  determine  which  decision  making  methodology.  Analytical  or  Naturalistic, 
is  most  appropriate  for  a  given  situation;  and,  (3)  identify  the  leading  cause  for 
unfavorable  outcomes  for  those  hazreps  resulting  in  an  unfavorable  conclusion. 
Accomphshing  these  objectives  will  provide  the  requirements  necessary  to  better  train 
and  equip  decision  makers  in  an  aeronautical  environment. 
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C.  THESIS  OVERVIEW 


Chapter  n  discusses  the  process  of  human  decision  making  and  the  need  for 
gaining  situational  awareness.  It  reviews  the  literature  to  present  both  Analytical  and 
Naturalistic  Decision  Making  perspectives  and  the  strengths  and  weaknesses  of  each. 
Several  models  of  decision  making,  situation  assessment  and  cognitive  appraisal  are 
presented  as  background  material. 

Chapter  III  outlines  the  methodology  used  for  analyzing  the  hazard  reports  for 
this  study.  A  brief  history  of  the  Naval  Aviation  Safety  Program  is  presented  to  give  the 
reader  insight  into  the  submission  procedures  and  content  of  a  hazrep.  The  author’s 
expert  credentials  are  reviewed  to  lend  credibility  to  the  analysis.  The  categorization 
schemes  of  the  hazrep  are  identified  and  explained. 

Chapter  IV  presents  the  results  of  the  analysis  and  discusses  the  applicability  and 
relevance  of  the  situations  encountered  to  the  two  decision  making  methodologies. 
Explanations  for  both  favorable  and  unfavorable  outcomes  are  given  and  discussed. 
Finally,  a  hybrid  model  of  decision  making  is  presented. 

Chapter  V  summarizes  the  findings  and  proposes  recommendations  based  on  the 
results.  The  recommendations  are  intended  to  both  increase  situational  awareness  among 
pilots  and  reduce  the  occurrence  of  erroneous  indications  fi'om  faulty  aircraft  systems. 
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n.  HUMAN  DECISION  MAKING 


A.  INTRODUCTION 

Decision  making  is  a  process  that  includes  not  only  the  decision  and  its 
subsequent  action,  but  also  the  initial  cues  indicating  that  a  decision  must  be  made  and 
assessments  of  the  environment.  After  stimuli  have  occurred,  the -decision  maker  must 
accurately  assess  the  surroundings  and  develop  situational  awareness  to  make  an  effective 
decision.  This  chapter  discusses  the  processes  involved  in  assessing  a  situation  and  the 
use  of  different  cognitive  strategies  to  make  a  decision.  A  representative  sampling  of 
models  is  presented  as  background  material  for  their  subsequent  synthesis  into  an 
aeronautical  decision  making  model. 

The  literature  on  decision  making  indicates  that  there  are  two  distinct  methods  of 
decision  making:  analytical  and  intuitive.  Anal3dical  methods  are  based  on  a  structured, 
logical  analysis  of  the  situation,  while  intuitive  decisions  are  based  on  pattern  recognition 
and  experience.  Intuitive  decision  making  is  commonly  referred  to  as  Naturalistic 
Decision  Making  (NDM),  which  has  its  roots  in  research  on  how  decisions  are  made  in 
actual,  non-laboratory  settings  by  experienced  people.  It  asserts  that  people  make 
decisions  based  on  recognizing  a  situation  as  similar  to  situations  stored  in  memory  and 
applying  responses  that  worked  well  to  these  situations  in  the  past  (Klein,  1986). 
Analytical  decision  making  relies  on  reasoning,  precise  information  and  timely 
calculations  (Wright,  1974)  while  NDM  relies  on  experience  and  judgment. 
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B.  OODALOOP 


When  discussing  decision  making  in  any  context,  it  is  often  helpful  to  model  the 
decision  making  process.  The  figure  below  illustrates  the  OODA  loop  as  modeled  by 
Colonel  John  R.  Boyd  in  A  Discourse  on  Winning  and  Losing,  which  is  discussed  in 
Naval  Doctrine  Publication  Six,  Naval  Command  and  Control  (1995).  While  normally 
thought  of  as  being  applicable  to  a  conflict  between  parties,  this  model  can  be  used  to 
describe  the  way  in  which  decision  makers  gather  information  to  reduce  uncertainty,  gain 
situational  awareness  and  make  decisions  regarding  a  course  of  action  in  any  situation. 
The  OODA  loop  is  an  iterative  process,  in  which  the  decision  maker  is  constantly 
engaged  in  collecting  information,  making  evaluations  and  decisions,  and  observing  the 
consequences  of  these  decisions.  The  acronym  stands  for  observe,  orient,  decide  and  act, 
which  describes  the  sequence  of  events  performed  by  a  decision  maker.  A  simple 
scenario  involving  a  minor  aircraft  malfunction  is  used  to  help  illustrate  the  process. 

Observe 

Act 

Decide 

Figure  2-1  OODA  Loop  (adapted  from  Naval  Command  and  Control) 
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1.  Observe 


The  first  step,  to  observe,  involves  gathering  information  on  the  situation  from 
sources  within,  or  external  to,  the  decision  maker's  situation  environment.  This 
observation  may  occur  as  a  result  of  a  stimulus  alerting  the  decision  maker  that  an  action 
is  required.  For  example,  the  initial  data  may  be  a  flashing  oil  pressure  indicator  light  or  a 
rise  of  oil  pressure  noted  from  a  gauge. 

2.  Orient 

The  second  step,  to  orient,  is  achieved  by  gathering  more  data  on  the  situation 
and  converting  it  to  useable  information,  knowledge  and  understanding.  The  nature  of 
the  malfunction,  seriousness  of  the  indications,  timeliness  of  the  actions  required,  and 
procedures  required  are  vital  pieces  of  information  the  decision  maker  needs  to  assess  the 
situation  and  form  a  mental  image. 

3.  Decide 

The  third  step  in  the  OODA  loop  is  making  a  decision,  which  is  choosing  a  course 
of  action  that  is  either  a  response  to  the  observations  or  a  plan  for  future  events.  For 
example,  a  Plane  Commander  decides  to  abort  the  mission  and  divert  to  an  alternate 
airfield  after  analyzing  the  situation  and  performing  the  applicable  procedures  for  a  rise  in 
oil  pressure. 

4.  Act 

Taking  action  is  the  fourth  step,  and  its  execution  is  monitored  by  the  decision 
maker  and  the  pertinent  crew  members.  In  the  case  of  an  aircraft  experiencing  a 
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malfunction,  this  monitoring  involves  both  internal  cues  and  external  observations  of  the 
environment  (in  the  example  above,  diverting  to  an  alternate  airfield  would  include 
communicating  with  FAA,  ATC,  airport  personnel,  etc.).  A  feedback  loop  exists  to 
ensure  the  action  taken  is  appropriate,  so  that  adjustments  can  be  made  to  alter  the 
decision,  as  more  information  is  gathered.  The  OODA  loop  is  a  cyclical  process  that 
requires  feedback  to  function  effectively. 

It  is  important  to  understand  that  many  of  these  OODA  loops  occur 
simultaneously.  While  a  Plane  Commander  is  gathering  information  and  observing  the 
situation,  the  Tactical  Coordinator  also  is  assessing  the  impact  of  the  malfunction  and  the 
impact  that  the  resulting  decision  will  have  on  the  mission.  At  the  same  time,  the  copilot 
and  flight  engineers  are  gathering  information  with  the  intention  of  either  making  their 
own  decision  or  providing  input  to  the  Plane  Commander.  An  action  initiated  by  any  of 
these  crew  members  may  affect  the  observations  of  the  Plane  Commander  and  therefore 
alter  his  OODA  loop  accordingly. 

C.  SITUATIONAL  AWARENESS 

As  can  be  seen  in  the  OODA  loop,  it  is  necessary  to  observe  and  orient  prior  to 
deciding  on  a  course  of  action.  During  this  phase  the  decision  maker  conceptualizes  the 
environment,  or  attempts  to  gain  situational  awareness.  Situational  awareness  is  formally 
defined  as: 

...the  perception  of  the  elements  in  the  environment  within  a  volume  of 
jime  and  space,  the  comprehension  of  their  meaning  and  the  projection  of 
their  status  in  the  near  future  (Endsley,  1988,  p.  36). 
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Endsley  states  that  decision  makers  often  have  an  internal  representation  or  a 
mental  model  comparable  to  the  situation  with  which  they  are  dealing.  This  mental  model 
is  compared  to  the  current  situation  and  projections  are  made  (based  on  the  degree  of 
match)  to  predict  events  as  they  should  occur  in  the  future.  This  is  the  basis  for  two  of 
the  situational  awareness  models  discussed  in  this  chapter.  A  more 'Specific  definition  of 
situational  awareness,  borrowed  fi'om  a  U.S.  Navy  Aircrew  Coordination  Training  (ACT) 
Course,  is:  "the  extent  to  which  a  pilot^s  perception  of  his  environment  mirrors  reality." 
(Patrol  Squadron  THIRTY  ACT  Course,  1991) 

Assessing  and  correctly  understanding  the  situation  frequently  poses  the  major 
portion  of  any  decision  making  task  related  to  a  malfunction  or  emergency  condition  in  an 
aircraft.  In  almost  any  situation,  effective  decision  making  depends  largely  on  having  a 
good  understanding  of  the  situation  and  one^  surroundings.  Too  often,  situation 
assessment  is  not  given  due  diligence,  and  situational  awareness  is  either  never  developed 
or  lost.  In  fact,  many  human  errors  attributed  to  poor  decision  making  can  be  traced  to 
errors  in  correctly  assessing  the  situation  and,  as  a  result,  taking  inappropriate  action 
based  on  this  initial  erroneous  situation  assessment.  An  example  of  incorrectly  assessing 
a  situation  is  an  aircrew  who  secures  a  perfectly  good  engine  as  a  result  of  an  erroneous 
turbine  inlet  temperature  indication,  believing  an  overly  high  temperature  condition  exists. 
While  securing  the  engine  would  have  been  the  correct  procedure  if  an  actual  overly  high 
temperature  existed,  in  this  case  it  is  unnecessarily  shut  down  and  places  the  crew  in  a 
potentially  hazardous  situation. 
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The  foflowing  discussions  explain  the  underlying  processes  and  mechanisms  in 
developing  situational  awareness  and  the  relationship  between  this  assessment  of  the 
environment  and  the  decision  making  process. 

1.  Cognitive  Hierarchy 

Numerous  Service  Publications  (e.g.  NDP  6,  Naval  Command  and  Control, 
Marine  Corps  Doctrine  Publication  6,  Command  and  Control)  refer  to  a  model  of  the 
Cognitive  Hierarchy  to  characterize  how  a  decision  maker  deciphers  information  to 
achieve  fiiU  understanding  of  a  situation.  Figure  2-2  (adapted  firom  NDP  6,  Naval 
Command  and  Control)  illustrates  the  four  steps  in  generating  this  understanding,  or 
situational  awareness,  and  can  be  viewed  as  the  process  used  during  the  first  two  steps  of 
the  OODA  loop,  observe  and  orient. 


Figure  2-2  CognitiTe  Hierarchy  (adapted  from  Naval  Command  and  Control) 


Data  is  meaningless  to  a  decision  maker  without  processing.  For  instance, 
electronic  signals  generated  by  a  tachometer  generator  on  the  reduction  gear  box  are 
simply  not  useable  until  they  are  processed  and  transformed  into  a  recognizable  and 
meaningful  format  or  display.  Once  this  processing  occurs,  the  data,  now  integrated  in 
the  form  of  information,  is  placed  in  its  situational  context  (RPM  readout  on  a  flight 
station  gauge).  Using  cognition  and  learned  aspects  of  the  imphcations  of  RPM  and 
propellers,  the  decision  maker  can  integrate  various  pieces  of  information  to  build  an 
informed  picture  of  the  situation.  Perhaps  an  audible  pitch  change  acconqranies  what 
appears  to  be  an  RPM  over  speed  condition  on  the  gauge.  This  analyzing  and  fusing  of 
various  pieces  of  information  serves  to  both  validate  the  information  as  factual  and  build 
a  clearer  picture  of  the  situation.  Finally,  judgment  is  applied  by  the  phot  to  transform 
the  knowledge  into  understanding.  Intuition,  experience,  expertise  and  training  are  aU 
relevant  to  judgment  and  understanding.  The  consequences  of  allowing  an  over  speed 
condition  to  exist  on  an  operating  engine  and  the  ramifications  of  acting  on  the  condition 
are  now  clear  to  the  decision  maker  as  understanding  emerges  from  knowledge. 

While  it  is  certainly  desirable  to  achieve  full  understanding  or  perfect  situational 
awareness  of  a  situation  before  making  a  decision,  it  is  obvious  that  many  decisions  must 
be  made  in  an  environment  of  ambiguous,  erroneous,  missing,  or  conflicting  information. 
To  make  matters  worse,  many  decisions  must  be  made  in  an  environment  constrained  by 
time-pressure  and  considerable  amounts  of  stress.  As  uncertainty,  ambiguity  and  stress 
increase,-  the  pilot  must  find  other  methods  to  synthesize  the  available  cues  into  an 
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accurate  understanding.  Experience  may  be  the  key  to  imderstanding  and  grasping  the 
situation  faster  and  more  reliably. 

2.  Cue  Recognition 

It  has  been  hypothesized  that  experienced  pilots  differ  from  inexperienced  pilots  in 
the  ability  to  retrieve  domain-specific  knowledge  representations  from  long-term  memory 
(Stokes,  Belger  &  Zhang,  1990).  Thus,  when  confronted  with -situations  that  could 
endanger  the  safety  or  efficiency  of  the  flight,  experienced  pilots  may  more  readily 
recognize  cues  relevant  to  the  problem.  They  cotrqrare  these  cues,  through  a  process 
referred  to  as  "pattern  matching,"  with  situational  schemas  stored  in  long-term  memory. 
Only  if  they  are  unable  to  match  cues  to  their  mental  maps  do  pilots  drop  into  an 
alternative  strategy  using  real-time  computational  and  inferential  processes  of  working 
memory  (Stokes,  Kemper,  &  Kite,  1997).  Inherent  in  working  memory  is  a  "thinking 
balloon"  of  cognitive  appraisal  that  may  expand  in  size  and  detract  from  working 
memory's  capacity  as  time-pressure,  stress  and  uncertainty  increase.  Figure  2-3,  adapted 
from  PC  Based  Instnment  Flight  Simulation  (Sadlowe,  1991),  illustrates  the  process  of 
pattern  matching  in  long-term  memory  versus  the  analytical  processing  done  in  working 
memory.  Most  of  a  personas  active  processing  of  information  must  occur  in  working 
memory.  New  information  must  be  combined  with  existing  knowledge  and  a  conqrosite 
picture  of  the  situation  developed  for  imderstanding  to  occur  (refer  to  Cognitive 
Hierarchy).  Projection  of  future  events  and  subsequent  decisions  regarding  appropriate 
courses  of  action  must  occur  in  working  memory  as  well.  For  inexperienced  decision 
makers,  or  those  dealing  with  unique  situations,  working  memory  may  constitute  the 
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No  Match 


forking  Memory 

^Controlled  Processing 
Integrate  Cues 
Generate  Original  Solutions 


Figure  2-3  Pattern  Matching  Model  (adapted  from  PC  Based  Instrument  Flight  Simulation) 

main  bottleneck  for  situational  awareness.  Within  this  model,  experienced  pilots  exhibit 
more  effective  and  efiBcient  decision  making  as  they  can  more  readily  identify  significant 
cues  and  gain  situational  awareness  than  can  the  pilot  who  must  integrate  cues  and 
generate  an  original  solution. 

3.  Noble:  Situation  Assessment 

Noble's  model  of  situation  assessment  is  similar  to  the  Cue  Recognition  model 
above.  Information  on  previous  experiences  is  stored  in  the  decision  maker's  memory  in 
order  to  form  an  interpretation  of  the  current  situation  (Noble,  1989).  The  process  of 
performing .  situation  assessment  unfolds  as  follows  (Figure  2-4):  first,  concrete 
information  about  the  situation  is  combined  -with  environmental,  or  "context"  information. 
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Then,  this  current  situation  assessment  is  compared  Avith  knowledge  retrieved  from  the 
decision  maker's  long-term  memory  to  form  a  tentative  interpretation  (representation)  of 
the  situation.  This  representation  includes  certain  expectations  concerning  future 
concrete  and  contextual  information.  These  expectations  are  tested  by  this  additional 
information  from  numerous  sources.  To  the  extent  that  the  expectations  do  not  match 
this  information,  the  representation  is  refined  or  rejected  in  favor  of  a  new  representation 
that  is  tested,  retained,  refined  or  rejected  in  turn.  Thus,  people  sometimes  assess  and 
decide  by  observing  how  the  current  situation  is  similar  to  previously  observed  situations 
and  implementing  actions  that  worked  in  these  previous  situations. 


Figure  2-4  Noble's  Situation  Assessment  Model  (1993) 
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D.  ANALYTICAL  DECISION  MAKING 

1.  Background 

Analytical  decision  making  can  provide  excellent  results  and  is  well  suited  for 
problems  that  have  an  accurate  information  base  with  clearly  defined  goals.  It  is  soundly 
based  in  logic  and  is  specifically  designed  to  eliminate  biases  and  intangible  or  qualitative 
factors  that  are  difficult  (if  not  impossible)  to  measure.  The  process  can  be  reduced  to 
algorithms  and  programmed  in  a  computer.  These  processes  can  be  easily  taught  and  are 
independent  of  the  knowledge  and  experience  of  the  decision  maker.  This  implies  that  a 
junior,  inexperienced  decision  maker  will  reach  the  same  answer  as  an  expert  if  they  have 
the  same  data  and  use  the  same  process.  Analytical  decision  methods  also  tend  to  give 
the  decision  maker  a  clear,  defined  rationale  to  justify  and  document  their  decisions, 
especially  when  rationalizing  an  optimal  solution  with  clear  goals. 

Analytical  decision  making  may  be  best  used  for  predeployment  and  preflight 
preparation  planning  rather  than  "heat  of  the  moment"  or  emergency  scenarios. 
Administrative  and  logistical  planning  scenarios  may  also  lend  themselves  to  analytical 
methods  to  best  optimize  resources. 

The  classical  model  of  analytical  decision  making  holds  that  it  is  a  rational  and 
systematic  process  of  analysis  based  on  the  concurrent  comparison  of  multiple  options. 
Bemoulh's  Maximization  of  Expected  Value  is  designed  to  choose  the  best  option  given 
ordinal  or  better  information.  For  each  option  the  product  sum  is  computed  across 
outcomes  and  the  option  with  the  largest  product  sum  (i.e.,  the  highest  expected  value)  is 
selected  (Sommer,  1954).  It  usually  requires  some  type  of  computational  aid,  demands  a 
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well  structured  problem  and  often  takes  extensive  time  for  application.  Preflight  planning 
for  a  transoceanic  evolution  is  an  ideal  opportunity  to  employ  an  expected  value 
methodology,  as  it  requires  selection  of  an  optimal  flight  path  given  a  multitude  of 
resources  and  constraints  (fuel,  weather,  field  availability,  winds  aloft,  etc.). 

Using  this  methodology  is  highly  time  consuming  in  both  the  computational 
aspects  and  the  option  generation  process.  For  the  procedure  to"perform  to  its  best 
capability,  an  exhaustive  option  production  process  must  first  be  conducted  to  ensure 
completeness  and  optimization  among  competing  alternatives.  It  takes  a  while  to 
identify,  analyze,  and  compare  all  the  various  alternatives  available.  As  a  result,  no 
matter  how  quickly  the  computations  can  be  performed,  there  will  always  be  a  certain 
minimum  amount  of  time  associated  with  a  decision  reached  by  this  method. 

The  analytical  model  also  requires  a  high  level  of  certainty  and  accuracy  of 
information.'  It  assumes  that  the  information  is  available  and  reliable  or,  if  it  is  not 
available,  that  the  decision  maker  will  have  the  ability  to  obtain  it.  This  may  impact  the 
amount  of  time  required  to  make  the  decision  as  it  will  certainly  take  time  to  gather  more 
information.  If  information  is  missing,  altered,  or  inaccurate,  the  resulting  quality  of  the 
decision  will  suffer. 

Experience,  perspective  and  judgment  are  not  requisites  of  the  analytical  model 
and  are  not  taken  into  consideration  when  computing  the  optimal  solution.  The  analytical 
model  is  process-based  and  arrives  at  the  correct  solution  as  long  as  the  right  information 
is  used  in  problem  formulation.  This  model's  failure  to  consider  expertise  of  the  decision 
maker  results  m  the  prediction  that  near-novice  pilots  using  analytic  decision  making 
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methods  can  operate  under  the  same  conditions  as  their  high-time  seasoned  peers.  The 
veteran's  years  of  experience  in  the  flight  station  add  no  particular  insight  to  picking  the 
optimal  flight  path  with  regard  to  winds  aloft  or  weather  avoidance. 

The  appeal  of  the  analytical  process  is  clearly  the  highly  defined,  clear  cut,  orderly 
process  that  it  offers.  Decisions  made  with  this  method  have  ample  documentation  to 
justify  and  rationalize  the  final  solution.  Decision  makers  have  at  their  disposal  a  history 
of  information  that  can  be  scrutinized  and  investigated  to  ensure  the  proper  decision  was 
indeed  made. 

In  general,  analytical  decision  making  is  more  appropriate  under  the  following 
conditions: 

-Time  is  not  a  fector. 

-A  choice  among  clearly  defined  options  exists. 

-Complete  and  reliable  information  is  available. 

-Decision  makers  lack  experience. 

-Decisions  must  be  justified. 

2.  Disadvantages 

The  biggest  disadvantages  of  using  analytical  decision  making  in  an  aeronautical 
environment  are  the  requirements  for:  (1)  sufficient  time  to  gather,  analyze  and  choose  a 
decision,  and  (2)  accurate  information  as  an  input  to  the  process.  It  is  often  time 
consuming  to  compile  and  analyze  the  vast  amounts  of  data  required  for  the  analytic 
decision  making  process.  Often,  the  information  required  to  make  the  decision  is  lacking, 
misleading  or  wrong.  Many  situations  encountered  in  an  aircraft  are  unique  to  the 
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particular  event,  so  the  information  required  for  the  option  generation  phase  is  corrpletely 
unknown.  Further,  to  improve  the  certainty  of  the  optimal  solution,  decision  maker’s 
must  gather  even  more  information,  which  can  increase  the  time  factor  considerably.  One 
study  also  suggests  that  analytical  decisions  are  shown  to  be  less  successful  than  other 
methods  when  made  under  time-pressure  and  stress  (Wright,  1974).  Since  the  analytical 
strategy  can  be  reduced  to  algorithms  and  decision  criteria,  the  use  Of  analytical  decision 
making  in  a  tactical  scenario  may  also  give  the  enemy  an  advantage,  if  they  were  to 
process  the  same  information  and  predict  the  decision  maker’s  actions. 

E.  NATURALISTIC  DECISION  MAKING 

1.  Background 

The  naturalistic  decision  making  (NDM)  model  is  derived  from  the  study  of  how 
experienced  people  actually  make  decisions  in  real-world  situations  or  in  "naturalistic" 
scenarios.  A  short  definition  of  NDM  is:  "the  way  people  use  their  experience  to  make 
decisions  in  field  settings."  (Zsambok,  1997,  p.  4).  The  naturalistic  decision  making 
movement's  center-of-giavity  is  based  on  the  evidence  that  evolved  from  research  on  fire 
ground  commanders  (Klein,  Calderwood,  Clinton-Cirocco,  1986).  Klein  et  al.'s  body  of 
research  essentially  states  that  experts  use  their  experience  to  adopt  a  successful  course 
of  action  without  applying  rigorous  analytical  strategies  or  deliberating  over  more  than 
one  option  prior  to  making  a  decision.  Various  studies  of  how  Naval  command  and 
control  officers  and  infentry  soldiers  decide  have  generated  the  same  conclusions:  "under 
operational  conditions,  decision  makers  rarely  use  analytical  methods,  and  non-analytical 
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methods  can  be  identified  that  are  flexible,  efiBcient  and  effective.”  (Klein  &  Klinger, 
1991).  Naturalistic  decision  making  strategies  are  not  suitable  in  every  situation  and  the 
conditions  where  they  are  most  applicable  include  dynamic  and  continually  changing 
conditions,  real  time  reactions  to  these  changes,  ill  defined  tasks,  time  pressure, 
significant  personal  consequences  for  mistakes,  and  experienced  decision  makers  (Klein, 
1993). 

Using  this  intuitive  process,  experienced  decision  makers  assess  the  situation  and 
try  to  recognize  familiar  patterns.  The  first  solution,  although  not  necessarily  optimal,  is 
generally  workable  and  evaluated  to  be  "good  enough".  Mental  simulation,  rather  than 
decision  analysis,  expected  value,  or  multi-attribute  utility  functions,  is  used  to  evaluate 
the  options.  The  primary  methods  used  by  decision  makers  employing  this  naturalistic 
strategy  are  pattern  matching  and  mental  simulation.  Pattern  matching  involves  a 
comparison  -of  the  current  situation  with  previously  experienced  situations.  In  complex 
situations,  mental  simulation  involves  applying  the  response  that  the  decision  maker  has 
decided  to  use  to  see  how  the  results  would  "play  out"  if  a  specific  response  is  taken. 

The  most  significant  advantages  of  using  intuition  for  decision  making  are  speed, 
creativity,  tolerance  of  uncertainty  and  the  ability  to  visualize  the  problem  and  its 
solution.  In  many  malfunction-oriented  aeronautical  decisions,  time  is  the  critical  factor. 
Intuitive  decisions  tend  to  be  much  faster  as  they  rely  on  selection  of  a  "satisficing" 
course  of  action  (i.e.,  hot  necessarily  optimal,  but  good  enough)  based  on  experience, 
rather  than  the  generation  and  comparison  of  a  multitude  of  possible  solutions  until  the 
optimal  one  is  chosen. 
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2.  Klein's  Recognition-Primed  Decision  Model  (RPD) 

Klein^  model  holds  that  experts,  who  are  pressed  for  time,  while  feeing  a  task 
loaded  with  uncertainty,  &st  assess  whether  the  situation  strikes  them  as  femiliar  or 
unique.  This  process  is  referred  to  as  "recognition-primed"  because  recognizing  the 
situation  as  similar  to  some  situation  previously  experienced  is  key  to  how  the 
accompanying  response  is  generated  and  the  ensuing  action  is  carried  out.  Typical 
situations  lead  to  typical  (procedural)  actions,  while  novel,  or  unique,  situations  pose  new 
challenges,  which  cannot  be  solved  simply  by  applying  the  same  routine.  To  recognize 
the  situation  and  guide  the  selection  of  proper  action,  the  decision  maker  identifies  critical 
cues  that  may  explain  the  situation  and  causal  fectors  that  explain  what  is  occurring  and 
what  may  occur  in  the  future.  Experts  use  prior  knowledge  and  experience  to  recognize 
and  classify  a  situation.  Rather  than  comparing  in  detail  the  pros  and  cons  of  the  different 
outcomes,  they  quickly  imagine  (via  mental  simulation)  how  one  course  of  action  may 
play  out  and  then  either  choose  to  adopt  it,  if  it  satisfies  the  situation,  or  adapt  the 
response.  Experts  take  the  first  workable  solution  they  can  find,  and  while  it  may  not  be 
optimal,  it  is  often  quite  good. 

When  the  situation  is  not  recognized  as  femiliar  or  typical,  the  experienced 
decision  maker  can  still  act.  They  can  modify  a  course  of  action,  retrieved  firom  memory, 
that  worked  in  a  similar  situation  to  accommodate  the  current  situation. 

Klein  emphasizes  that  RPD  is  not  a  universal  model  of  decision  making.  It  is  a 
model  that  is  more  suitable  for  situations  that  are:  (1)  characterized  by  high  levels  of 
uncertainty,  and  (2)  likely  to  be  encountered  under  time  pressure  by  people  with  high 
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levels  of  expertise.  It  is  less  likely  to  be  applicable  if  these  conditions  are  not  met,  if  the 
decisions  are  naturally  presented  as  choices,  and  if  the  decision  maker  feels  a  need  to 
optimize  the  decision  (Lipshitz,  1993). 

3.  Montgomery's  Search  for  Dominance  Structure 

Montgomery  is  interested  in  how  decisions  are  actually  made  when  several 
alternatives  are  available.  His  answer  is  simply  that  a  decision  maker  will  search  for  a 
dominant  alternative  (Montgomery,  1989).  An  alternative  is  said  to  be  dominant  if  it  is  at 
least  as  attractive  as  its  competitors  on  all  relevant  attributes,  and  exceeds  each  of  them 
on  at  least  one  attribute.  Essentially,  Montgomery  thinks  of  decision  making  as  the 
process  of  finding  a  good  argument  for  acting  in  a  certain  way,  first  by  a  quick  selection 
of  a  promising  alternative  and  then  by  testing  or  ensuring  the  dominance  of  this 
alternative.  He  feels  this  is  compatible  with  the  limited  capacity  of  human  information 
processing,  by  focusing  on  a  limited  number  of  alternatives  and  accentuating  the 
differences  between  them.  Also,  the  availabihty  of  a  dominant  alternative  helps  decision 
makers  to  persist  in  its  implementation  and  to  rationalize  and  justify  their  choice.  A 
drawback  of  this  persistence  in  implementation  may  be  the  distortion  of  reality  to 
maintain  dominance  of  an  alternative  that  is  actually  dominated  by  a  better  choice  when 
the  environment  changes  (Montgomery,  1993). 

4.  Rasmussen's  Decision  Processes 

Rasmussen  is  interested  in  the  decision  making  processes  of  human  operators  of 
complex  systems.  He  has  distinguished  between  three  types  of  behavior  that  are 


21 


controlled  by  qualitatively  diflFerent  cognitive  mechanisms:  skill-based  behavior,  rule- 
based  behavior,  and  knowledge-based  behavior  (Rasmussen,  1983). 

Skill-based  behavior  includes  expert  sensory-motor  performance,  which  runs 
smoothly  and  efficiently  without  conscious  attention.  Skill-based  behavior  is  controlled 
by  a  dynamic  mental  model  that  depicts  the  decision  maker's  movements  and  environment 
in  real  time,  thereby  enabling  him  to  adjust  rapidly  to  feedback.  An'example  of  this  type 
of  behavior  is  the  pilot  who  subconsciously  applies  slight  back  pressure  to  the  yoke  to 
prevent  a  loss  in  altitude  as  a  minor  nose  down  pitch  occurs  in  the  aircraft.  This 
correction  is  automatic  and  taxes  neither  long  term  nor  working  memory. 

Rule-based  behavior  is  controlled  by  rules  and  procedures  that  can  be  stated 
explicitly  by  the  decision  maker.  This  is  a  learned  behavior  and  can  become  skill-based  if 
practiced  and  repeated  often  enough.  Information  at  this  level  is  easily  processed  and 
indicates  that  a  situation  exists  that  requires  invoking  a  known  rule  or  specific  action. 
Noble's  situation  assessment  and  Klein's  RPD  depict  rule-based  behavior  in  the  early 
stages  of  situation  recognition  (Lipshitz,  1993).  Pulling  the  emergency  shut  down  handle 
for  an  indication  of  an  engine  fire  is  exhibiting  rule-based  behavior. 

Knowledge-based  behavior  requires  a  deeper  understanding  of  the  situation 
before  any  action  may  be  taken.  An  explicit  consideration  of  options  and  objectives  may 
be  accomplished  to  preclude  inappropriate  behavior  for  a  misunderstood  environment. 
Unique  situations,  or  environments  with  uncertainty  and/or  ambiguity  are  cases  where 
knowledge-based  behavior  is  most  likely  encountered.  In  the  case  of  the  engine  fire 
above,  rule-based  behavior  may  move  up  a  level  to  knowledge-based  if  the  fire  is 
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accompanied  by  a  simultaneous  malfunction  on  another  operating  engine.  An  analysis  of 
alternatives  and  ramifications  of  action  would  be  conducted  prior  to  automatically 
invoking  a  procedure. 

5.  NDMUse 

Ambiguous,  unstructured  environments  may  be  the  best  domains  in  which 
Naturalistic  Decision  Making  is  useful.  Any  situation  requiring  a  rapid,  almost 
subconscious  decision  may  also  be  best  suited  for  an  intuitive  process  rather  than  a 
lengthy  analytical  one.  Generally,  intuitive  decision  making  is  better  in  conditions  where: 

-Time  is  critical. 

-Decision  makers  are  knowledgeable  and  experienced. 

-There  is  a  high  degree  of  uncertainty. 

-There  is  ambiguous  or  changing  information. 

-Innovative  or  creative  thought  is  required. 

The  essential  factor  in  NDM  is  experience.  Experience  is  the  critical  element  that 
allows  for  the  situation  assessment  (which  leads  to  situational  awareness)  that  is  at  the 
heart  of  naturalistic  decision  making.  Experience  allows  the  decision  maker  to  recognize 
a  situation  as  typical,  or  if  not  typical,  provide  enough  information  firom  a  similar 
situation  to  turn  knowledge  into  understanding.  If  sufficient  experience  exists  then 
valuable  time  is  saved  by  not  having  to  integrate  cues  and  derive  an  original  solution. 

6.  Disadvantages 

There  are  drawbacks  to  using  NDM  for  important  aeronautical  decisions. 
Intuitive  decisions  are  based  upon  the  decision  maker’s  personal  experiences  and  are 


23 


subject  to  individual  biases,  prejudices,  level  of  experience,  and  ego.  The  decision  will  be 
based  on  memories  that  may  be  inaccurate,  incomplete  or  influenced  by  personal 
preferences.  The  situations  where  intuitive  thinking  is  thought  to  be  most  apphcable, 
under  high  stress  and  time  critical,  may  tend  to  inhibit  the  intuitive  process.  Naturalistic 
decisions  may  not  be  trusted  by  seniors,  peers  or  subordinates,  as  there  is  little  or  no 
documentation  to  back  them  up.  Finally,  inexperienced  decision  msricers  are  not  able  to 
make  good  intuitive  decisions,  as  they  have  little,  or  no,  experience  to  draw  from. 

F.  SUMMARY 

From  Bernoulli's  Expected  Value  computation  to  today's  NDM  movement, 
decision  theorists  have  produced  a  multitude  of  hj^potheses  on  how  and  why  decisions  are 
actually  made.  Each  strategy  has  roots  in  some  common  logic,  but  each  is  unique  in  its 
attempt  to  explain  the  reasoning  and  cognitive  strategies  of  the  decision  maker  for  a 
particular  set  of  circumstances  or  situation.  Whether  an  analytical  or  naturalistic  strategy 
is  used,  all  decision  makers  use  a  process  that  requires  a  stimulus,  developing  situational 
awareness,  choosing  a  decisive  course  of  action  and  implementing  this  action.  Situational 
awareness  is  the  most  critical  aspect  of  any  decision,  as  the  lack  of  it  (or  an  incorrect 
situation  assessment)  may  result  in  an  inappropriate  response  to  a  misdiagnosed 
malfunction. 

It  is  important  to  realize  that  while  cognitive  strategies  from  the  analytical  and 
naturalistic  methodologies  may  appear  to  be  mutually  exclusive,  they  can  be  used  in  a 
complimentary  fashion  during  a  single  decision  situation.  To  incorporate  an  analytical 
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optimization  into  a  decision  process,  after  an  intuitive  situational  assessment  has  been 
performed,  is  a  practical  method  to  deal  with  a  unique  situation  that  is  not  time  critical. 
For  example,  the  crew  that  now  has  time  to  conduct  an  optimal  alternate  airfield  search 
after  conducting  a  precautionary  engine  shut  down  for  an  unknown  vibration  is 
combining  the  two  methodologies.  The  experienced  crew  decides  the  vibration  warrants 
securing  the  engine,  but  now  has  the  time  to  afford  a  deliberate  ancftcalculated  choice  of 
alternative  landing  sites.  For  any  situation  it  is  important  to  consider  both  models  and 
what  combination  of  the  two  will  work  best. 
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m.  METHODOLOGY 


This  study  analyzes  P-3  hazard  reports  (hazreps)  to  identify  different  types  of 
decisions  and  decision  making  strategies  used  in  an  aeronautical  environment.  The 
following  sections  explain  the  purpose  of  a  hazrep,  the  format  and  information  contained 
within,  and  the  analysis  performed. 

A.  HAZARD  REPORTS 

1.  Bacl^round 

Unfortunately,  naval  aviation  mishaps  are  an  inevitable  result  of  operations 
involving  flying.  The  Navy  continues  to  strive  for  a  reduced  aviation  mishap  rate  through 
numerous  safety  programs.  The  Naval  Aviation  Safety  Program  is  such  a  program;  it  is 
designed  to  reduce  aircraft  accidents  through  careful  investigation,  documentation  and 
analysis  of  hazards.  The  goal  of  the  program  is  to  identify  and  eliminate  hazards  before 
they  contribute  to  aviation  mishaps.  A  hazard  is  defined  as  "a  potential  cause  of  damage 
or  injury  to  aircraft  or  personnel"  (OPNAVINST  3750.6Q,  1989,  p.44).  The  Navy's 
mishap  rate  of  50  mishaps  per  100,000  flight  hours  in  the  1950's  has  dropped  significantly 
to  0.7  per  100,000  flight  hours  in  FY  1999  due  mainly  to  this  ongoing  identification  and 
reporting  of  hazards  (Naval  Safety  Center  statistics,  2000). 

The  most  efficient  and  direct  detection  of  hazards  often  is  accomplished  by  an 
individual  or  command  with  first  hand  knowledge  of  the  circumstances  surrounding  a 
hazardous  situation.  All  command  safety  programs  have  methods  in  place  to  identify 
hazards,  such  as  reviews  of  operating  procedures,  safety  surveys,  analysis  of  equipment 
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failures  and  inflight  incident  reporting.  Material/equipment  failure,  human  error, 
physiological  episodes,  near  mid-air  collisions  and  bird  strikes  are  examples  of  the 
multitude  of  inflight  incidents  where  hazards  can  be  identified  and  reported. 

The  Aviation  Safety  Officer  (ASO)  is  a  designated  naval  aviator  or  naval  flight 
officer  who  is  a  graduate  of  the  Aviation  Safety  School.  The  ASO  is  a  participating  pilot 
or  naval  flight  officer  in  the  squadron  and  possesses  additional  training  as  a  specialist  in 
the  Naval  Aviation  Safety  Program.  For  the  purposes  of  investigating  and  reporting 
hazards,  the  ASO  is  the  primary  functionary  in  the  process  and  authors  the  hazrep  after 
receiving  appropriate  input  fi'om  squadron  members.  It  is  the  responsibility  of  the  ASO  (in 
conjunction  with  the  Commanding  Officer)  to  ensure  that  hazreps  are  submitted  not  only 
in  the  correct  format  but  with  full  and  forthright  information  surrounding  the  hazardous 
situation. 

2.  Format 

Once  a  hazard  is  identified,  submission  procedures  found  in  OPNAVINST 
3750.6Q  are  followed  to  publish  a  hazrep,  which  is  addressed  to  all  pertinent  commands, 
details  the  circumstances  under  which  the  hazard  was  identified  and  proposes  actions  to 
eliminate  its  future  occurrence.  To  ensure  the  widest  dissemination,  the  message  is 
addressed  to  offices  of  the  Chief  of  Naval  Operations,  Naval  Safety  Center  and  all 
activities  that  fly  P-3's.  The  ASO  assigns  risk  assessment  codes  to  the  hazard,  indicating 
the  probability  of  future  occurrence  and  potential  impact  of  the  hazard  if  it  does  occur. 

The  body  of  the  message  sununarizes  the  circumstances  surrounding  the  hazard 
and  its  specific  identification  along  with  a  listing  of  pertinent  aircraft,  equipment  and 
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weather  data.  The  body  of  the  hazrep  attempts  to  explain  the  what,  where,  why  and  how 
of  the  hazard  and  provide  enough  details  for  others  to  folly  comprehend  the  severity  and 
repercussions  if  the  hazard  were  to  occur  in  the  foture.  It  discusses  the  crew's  detailed 
analyses  in  order  to  give  the  reader  insight  into  the  pilot's  decision  making  and  thought 
processes.  The  corrective  action  section  may  propose  a  solution  to  eliminate  the  hazard 
(if  one  is  readily  apparent)  without  regard  to  cost  or  feasibility.  T^though  a  corrective 
action  to  eliminate  the  hazard  may  not  always  be  readily  identified,  the  hazrep  still  can  be 
used  as  a  valuable  learning  tool  for  pilots  presented  -with  similar  situations  in  the  foture. 
Commanding  Officer's  comments  conclude  the  message  and  provide  a  potential  forum  for 
further  discussion  of  the  severity  of  the  hazard  and  the  crew's  performance  in  handling  the 
situation.  Figure  3-1  shows  a  sample  format  of  a  general  use  hazrep. 

As  mentioned  above,  the  body  of  the  message  contains  the  relevant  facts 
surrounding  the  identified  hazard  and  the  situation.  The  information  presented  to  the  crew 
from  internal  and  external  sources,  the  decisions  made  by  the  pilot  and  the  actions 
performed  are  all  outlined  in  the  evidence  and  analysis  section.  This  process  mirrors  the 
OODA  loop  discussed  in  the  previous  chapter:  the  pilot  observes  the  environment,  orients 
the  information  into  some  useable  schema,  decides  on  a  course  of  action  and  then  executes 
this  course  of  action  after  the  decision  is  made. 

3.  Concerns  About  Quality  and  Content 

The  quality  of  hazreps  depends  on  the  quality  of  the  investigation  into  attendant 
circumstances.  The  success  of  the  hazrep  program  relies  on  the  submission  of  complete, 
open  and  forthright  information  and  opinions  in  hazreps.  The  exercise  of  command 
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influence  to  edit,  modify,  or  in  any  way  censor  the  content  of  reports  is  contrary  to  the 
spirit  of  the  program  and  is  prohibited  by  OPNAVINST  3750.6Q.  Unfortunately,  some 
Commanding  Officers  may  believe  that  truly  confessing  all  the  bad  with  the  good  may 
harm  the  squadron's  reputation  or  jeopardize  future  career  opportunities;  therefore  some 
hazreps  suffer  from  a  lack  of  completeness. 

Another  area  of  concern  is  that  hazreps  are  not  considered  privileged  information. 
Privileged  information  assures  confidentiality  of  the  information  given;  such  safety 
information  cannot  be  used  as  evidence  for  other  investigations  regarding  legal  matters, 
misconduct,  blame,  etc.  The  authority  for  granting  an  assurance  of  confidentiality  is 
strictly  limited  to  the  investigation  and  reporting  of  mishaps,  and  not  hazards.  The 
instruction  states  that  extreme  care  must  be  taken  to  prevent  giving  the  impression  that  the 
use  of  the  hazrep  is  for  safety  purposes  only,  as  is  the  case  with  mishap  investigations. 
This  lack  of  confidentiality  may  prevent  some  crews  from  giving  open  and  forthright 
information  from  fear  of  repercussions. 

B.  EXPERT  ANALYSIS 

Due  to  the  inherent  ambiguity  and  incompleteness  surrounding  some  of  the  content 
in  the  body  of  the  hazrep  and  the  Commanding  Officer's  comments,  and  due  to  the  lack  of 
formal  investigation  that  accompanies  the  situation,  the  research  method  requires  that  an 
expert  on  the  subject  conduct  the  analysis.  Such  expertise  is  required  to  decipher  the 
nuances  familiar  to  the  P-3  community,  fill  in  unwritten  gaps  when  ijuestions  arise  and 
provide  an  estimate  of  the  severity  of  a  particular  malfunction.  The  ability  to  extract  what 
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Message  Originator 

PATRON  XXXXXX//10// 

Addressees 

CNO  WASHINGTON  DC//N889E// 

CMC  WASHINGTON  DC//A/SD// 

COMNAVSAFECEN  NORFOLK  VA//00/10/1 1/054// 

ALL  ORION  ACnVITIES 

Header  &  subject  line 

UNCLAS  FOUO//N0370// 

SUBJ/nnS  IS  a  general  use  aviation  hazard  report.  Originator,  Serial  Number, 
Date,  P-3,  Bureau  number,  REPORT  SYMBOL  3750-19.//  - 

Reference 

REF/A/DOC/OPNAVINST  3750.6Q/-// 

Other  references  as  applicable 

Summary 

RMKS/1.  THIS  REPORT  CONTAINS  A  (Routine  or  Severe)  HAZARD  TO  NAVAL 
AVIATION.  ENDORSMENT  (Is  or  Not)  REQUIRED.  SUMMARY:  Describe  circumstances  in 
brief  statement  of  Who  did  What  or  What  component  failed  When. 

Message  Body 

2.  DATA 

A.  AIRCRAFT  (1)  Model,  (2)  Bureau  number,  (3)  Modex  and  side  number,  (4)  Reporting 
Custodian 

B.  EQUIPMENT  Nomenclature  of  aircraft  parts  or  other  parts  involved  (1)  Model,  (2)  Make, 
(3)  Part  number,  (4)  Equipment  code,  (5)  Configuration,  (6)  Work  nit  code,  (7)  etc. 

C.  ENVIRONMENT  List  (1)  Date,  (2)  Loc^  time,  (3)  Local  time  zone,  (4)  Day  or  Night,  (5) 
Location,  (6)  Altitude,  (7)  Weather,  (8)  Other  backgrormd  information 

3.  CIRCUMSTANCES 

A.  EVIDENCE  AND  ANALYSIS.  Fully  and  clearly  describe  the  hazard  in  terms  of  who  did 
what  or  what  component  foiled. 

B.  CONCLUSIONS.  Explain  how  hazard  could  result  in  damage  and/or  injury. 

4.  CORRECTIVE  ACTION.  Describe  corrective  action  taken  to  eliminate  the  hazard,  or 
corrective  action  needed  to  eliminate  the  hazard.  Or,  if  it  is  b^ond  the  capability  of  the  originator 
to  formulate  recommended  corrective,  so  state. 

5.  REMARKS.  Content  is  at  the  discretion  of  the  originator.  Maintenance  actions  required  to 
repair  failed  component  or  other  postflight  examination  results  can  go  here. 

6.  POINT  OF  CONTACT.  Name,  rank,  phone  number  for  inquires. 

7.  COMMANDING  OFFICER  COMMENTS.  Commanding  Officer's  endorsement  of  the 
report 
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may  appear  to  be  inconsequential  information  may  provide  valuable  insights  to  the 
resultant  decision. 

Based  on  previous  experience,  flight  hours  and  schooling  the  author  qualifies  as  an 
expert  for  the  purposes  of  this  study.  With  over  3500  flight  hours  in  the  P-3,  he  has 
personally  witnessed  many  of  the  hazards  identified  in  the  438  hazreps  used  in  this 
analysis.  As  a  former  Fleet  Replacement  Squadron  (FRS)  instructor;  NATOPS  instructor 
pilot  and  instmctor-under-training  (lUT)  instructor  pilot,  he  is  extremely  well  versed  in  the 
decision  making  processes  of  pilots  and  of  the  common  errors  associated  with  task 
overload,  loss  of  situational  awareness  and  poor  airmanship.  He  was  instrumental  in 
developing  the  Aircrew  Coordination  Trmning  (ACT)  course  for  the  P-3  FRS  and  taught 
the  training  track  through  several  iterations.  He  is  also  a  graduate  of  the  Naval  Aviation 
Safety  Officer  course  and  has  served  in  billets  as  Aviation  Safety  Officer  (ASO)  and  Safety 
Department  Head.  As  such,  he  has  authored  well  over  a  himdred  hazreps  (and  reviewed 
hundreds  more)  in  his  career. 

C.  CONTENT  ANALYSIS 

The  hazrep  was  divided  into  several  different  levels  based  on  type,  class  and 
category  as  shown  in  Table  3-1.  The  analysis  was  based  primarily  on  the  content  of  the 
evidence  and  analysis  paragraph,  with  supplemental  and  corroborating  evidence  from  the 
CO  comments  and  environmental  data  sections.  While  the  table  shows  the  different 
categories.  Figure  3-2  illustrates  the  process  used  to  extract  the .  data  into  useable 
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Type 

Class 

Category 

Knowledge 

Ambiguous  Indications 

Compound  factors 

Multiple  malfimctions 

Unique;  no  NATOPS  guidance 

Unusual  malfimction 

Rule 

Ambiguous  Indications  (single  COA) 
Compound  factors  (procedural) 

Multiple  malfimctions  (procedural) 
Simple  procedural  ” 

Unique;  no  NATOPS  guidance 

Unusual  malfunction 

Skill 

Improper  procedures 

Situational  awareness 

Siuvival 

Table  3-1  Hazrep  Type,  Class  and  Category 


information  for  the  decision  making  analysis.  As  can  be  seen,  after  the  type,  class  and 
categories  are  established,  each  hazxep  is  subjected  to  analysis  based  on  the  use  of 
judgment,  the  outcome,  and  an  explanation  for  the  outcome. 

1.  Type 

Classification  by  type  is  accomplished  by  selecting  the  aircraft  system  that  is  most 
responsible  for  the  submission  of  the  hazrep.  Table  3-2  shows  the  eight  different 
classifications  by  type  and  the  subsystems  within  each  type.  This  classification  by 
equipment  type  is  necessary  to  aid  in  the  identification  of  specific  systems  that  may  have  a 
greater  propensity  toward  failure,  thus  pinpointing  an  area  for  concentrating  engineering 
solutions  to  prevent  future  hazards.  This  also  may  shed  some  light  on  the  type  of  system 
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Figure  3-2  Methodology  Model 


malfunctions  that  are  more  prone  to  a  specific  kind  of  decision  making,  or  systems  that  are 
more  susceptible  to  errors  due  to  their  complexity,  ambiguity,  or  lack  of  current  guidance. 

Classification  by  type  is  a  relatively  straight  forward  process  and  is  usually  easily 
discernible  from  the  subject  line  or  opening  statement  of  the  summary  section.  Where 
multiple  components  from  different  systems  are  involved  in  the  scenario  (e.g.  chips  light 
and  propeller  malfunction),  the  system  that  is  most  responsible  for  the  subsequent 
decisions  and/or  procedures  is  cited  as  the  type.  While  numerous  systems  may  be  evident 
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in  a  single  hazrep,  one  of  them  is  usually  the  cause  for  the  majority  of  decision  making 
behavior.  For  example,  a  prop  overspeed  occurring  on  the  return  transit  after  an  oil  leak 
caused  a  mission  abort,  would  be  classified  as  engine,  since  the  decision  to  act  on  the  leak 
already  has  occurred. 


Type 

Malfunctions  pertaining  to: 

Airframe 

Hydraulic  system 

Flight  controls 

Windshield 

Landing  gear 

Brakes 

Flaps  ” 

Anteimas  (external) 

Vibrations  (not  engine  or  e-handle) 

Fuel  tanks 

Electrical 

Generators 

Circuit  breakers 

Fire  of  Unknown  Origin  (FOUO) 

Shock 

Engine 

Turbines 

Fire  warnings 

E-handles 

Bleed  air 

Flameouts 

Fuel 

Gear  boxes 

Oil  pressure  &  quantity 

Loss  of  power 

Vibrations 

Chips  lights 

Start/Restart 

Power  levers 

Instrument 

Navigation 

Attitude  source 

Gauges 

Near  mid-air  collisions 
(NMAC) 

Other  aircraft 

UAVs 

Vehicles 

Other 

Bird  strikes 

Basic  skill  errors 

Weather 

Pressurization 

Engine  driver  compressors  (EDC) 
Pressurization  controller 

Air  multiplier 

Refrigeration  unit  turbines 

Outflow  valve 

Physiological  episodes 

Auxiliary  power  unit  (APU) 

Propeller 

Off  speed  conditions 

Pitchlock 

Decouple 

Fails  to  feather 

Fails  to  unfeather 

Negative  torque  sensing  system  (NTS) 

Feather  pump 

Prop  leaks 

RPMflux 

beta  &  NTS  inop  lights 

Prop  deice 

Prop  pump  lights 

Table  3-2  Classification  by  Type 
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2.  Class 


Sorting  the  hazreps  by  class  is  a  bit  more  difficult  than  separation  by  type.  This 
process  requires  grouping  the  hazreps  based  on  their  environment,  using  the 
characteristics  of  decision  processes  identified  by  Rasmussen  (1983):  knowledge-based, 
rule-based  or  skill-based  behavior.  "Behavior"  in  this  analysis  may  be  appended  with 
"environment,"  as  the  classification  is  based  on  the  circumstances  presented  to  the  decision 
maker  as  well  as  the  resulting  behavior.  However,  this  is  a  fine  distinction  as  it  is  almost 
certain  that  a  knowledge-based  environment  requires  knowledge-based  behavior,  since  the 
situation  requires  a  deeper  understanding  and  explicit  consideration  of  options  and 
objectives. 

The  following  definitions  apply  to  each  of  the  classes: 

Knowledge-based  -  requires  a  thorough  understanding  of  the  situation  before  any 
action  may  -be  taken;  multiple  alternatives  may  be  available  depending  on  situation 
assessment  and  judgment  of  the  decision  maker. 

Rule-based  -  rules  and  procedures  can  be  stated  explicitly  by  the  decision  maker; 
specific  action  is  required  for  given  cues  and  indications. 

Skill-based  -  evidence  of  expert  sensory-motor  performance  without  conscious 
attention  (automatic  stimulus-response  reactions);  skill  errors. 

Note  that  the  skill-based  environment  is  broadened  fi’om  merely  subconscious 
sensory-motor  actions  to  one  that  includes  immediate  "survival"  reactions  (near  mid-air 
collision  avoidance)  and  skill-based  errors.  These  survival  reactions  are  automatic 
stimulus-response  reactions  that  are  so  well  learned  that  they  appear  to  be  automatic. 
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Skill-based  errors  include  those  hazards  that  are  a  direct  result  of  improper  procedures 
being  performed  (or  proper  procedures  not  being  performed)  and  should  not  have 
occurred.  An  example  of  this  situation  is  a  three  engine  landing  for  an  oil  leak  that  is 
caused  by  an  improperly  seated  oil  cap.  The  skill-based  error  occurs  during  preflight  when 
the  cap  is  improperly  replaced  and  this  is  not  noticed. 

The  following  sections  develop  these  definitions  of  knowle'dge-based,  rule-based 
and  skill-based  classes  further  as  the  categories  within  each  are  defined.  The  most  difficult 
part  of  the  classification  is  discriminating  between  knowledge-based  and  rule-based  when 
the  situations  become  more  complex.  The  main  distinction  is  based  on  whether  there  is  a 
single  course  of  action  and  whether  the  decision  maker  has  to  use  experience  and 
extensive  judgment  in  the  decision  making  process. 

3.  Categories  within  the  Classes 

a)  Knowledge-based  Categories 

Table  3-3  shows  the  five  categories  within  the  knowledge-based  class  and 
the  accompanying  classifying  criteria.  The  categories  are:  ambiguous  indications, 
compound  factors,  multiple  malfunctions,  unique  situations  with  no  NATOPS  guidance 
and  unusual  malfunctions.  The  hazreps  fitting  these  categories  are  those  involving 
ambiguous  data  or  cues,  multiple  course  of  action  scenarios  where  the  pilot  must  make  a 
judgment  or  choose  among  many  alternatives,  and  unique,  or  unusual,  situations  that 
cannot  be  solved  vnth  a  simple  application  of  NATOPS  procedures. 
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b)  Rule-based  Categories 

Table  3-4  shows  the  six  categories  within  the  rule-based  class  and  their 


accompanying  classifying  criteria.  The  categories  are  essentially  the  same  as  the 
knowledge-based  categories  with  the  addition  of  simple  procedural. 

The  distinction  between  the  knowledge-based  and  rule-based  categories 
with  the  same  names  lies  in  the  situation.  If  the  situation  is  one  that-ean  be  handled  with  a 
strict  set  of  rules  or  procedures,  then  it  is  rule-based,  even  in  the  event  of  compound  or 
multiple  malfunctions.  If  the  resulting  action  is  the  same  regardless  of  the  number  of 
malfunctions  or  complexity  of  the  situation,  then  it  remains  rule-based.  For  example,  a  fire 
warning  that  is  not  evidenced  by  an  aircraft  system  warning  but  is  seen  visually  by  a  crew 
member  who  notices  a  torch  emitting  fi’om  the  turbine  (ambiguous  indications)  is  classified 
as  rule-based,  since  the  procedure  is  to  secure  the  engine.  On  the  other  hand,  a  fire 
warning  from  the  aircraft  system  that  continues  after  the  engine  is  secured,  but  is  not 
accompanied  by  visual  or  other  indications,  is  classified  as  knowledge-based,  since  the 
decision  to  release  subsequent  fire  extinguishing  agents  is  left  to  the  pilot's  discretion. 
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Table  3-3  Categories  in  Knowledge-Based  Class 
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Table  3-4  Categories  in  Rule-Based  Class 


c)  Skill-based  Categories 

Table  3-5  shows  the  three  categories  within  the  skill-based  class  and  their 
accompanying  classifying  criteria.  The  categories  are:  improper  procedures,  situational 
awareness  and  automatic  stimulus-response.  The  improper  procedures  and  situational 
awareness  categories  relate  to  errors  made  that  are  a  direct  cause  of  the  hazard.  The 
resulting  malfunction  would  not  have  occurred  if  the  person  or  persons  responsible  for  the 
error  had  properly  performed  the  procedures.  These  were  special  case  categories  created 
to  examine  how  skill  based  behaviors  affect  decision  making  and  record  how  often  they 


are  responsible  for  hazards. 


SkiU 

Categories 

Scenarios  involving: 

Sample  CO  comments 

Improper 

procedures 

Actions  of  the  pilot  or  crew  cause  a  malfimctinn  to  occur 
(e.g.  high  angle  of  attack  on  climbout  in  a  heavy  aircraft 
with  high  power  settings  causes  fire  warning) 

Never  should  have 
occurred... 

Complacency  and 
improper  procedures 
caused... 

Oversight  or  preflight  errors  cause  malfunction  (e.g.  oil  cap 
improperly  replaced  after  servicing,  brakes  not  set) 

Normal  inflight  procedures  performed  incorrectly  (e.g. 
improper  fuel  crossfeed  results  in  fuel  starvation 
and  flameout  of  engine,  altimeter  conversion) 

Chain  of  events  should 
have  been  broken. . . 

Situational 

awareness 

Unaware  of  surroundings  (e.g.  improper  malfunction  set  up 
in  training  scenario,  missed  conuns,  gauge  reading  error) 

SA  could  have 

Prevented... 

Unaware  of  implications  on  system  (e.g.  failure  to 

Ajfiects  of  loss  of  mag  heading  to  inertials) 

Auto  stimulus- 

response 

Immediate  evasive  action  (e.g.  NMAC  without  time  to  judge 
Closure  or  distance,  vehicle  on  runway) 

Pilofs  quick  reaction... 
Excellent  reflexes . . . 

Table  3-5  Categories  in  Skill-Based  Class 

4.  Judgment 


The  question  as  to  whether  the  situation  requires  judgment  on  the  part  of  the  pilot 
is  slightly  subjective.  If  the  pilot  uses  discretion  in  the  application  and/or  timing  of 
procedures  or  is  presented  a  choice  of  actions,  then  judgment  is  considered  to  have  been 
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used.  If  the  pilot  has  a  decision  regarding  a  best  case/worst  case  scenario,  then  judgment 
is  considered  to  have  been  used.  An  example  of  this  is  the  scenario  in  which  an  unknown 
fluid  leak  appears  on  the  engine  nacelle.  If  there  are  no  indications  on  the  flight  station 
gauges  to  confirm  the  leak,  the  pilot  may  reason  that  the  worst  case  is  a  fuel  leak  that 
presents  the  possibility  of  fire,  while  the  best  case  is  either  condensation  or  an  insignificant 
amount  of  oil.  It  is  then  up  to  the  pilot's  discretion  regarding  what  course  of  action  to 
take  —  secure  the  engine  and  abort,  continue  engine  operation  and  abort,  or  continue 
engine  operation  and  the  mission.  Judgment  is  not  limited  solely  to  the  knowledge 
category,  as  some  rule-based  scenarios  require  the  use  of  judgment  when  deciding  on 
aborting  the  mission,  possible  alternate  airfields  or  the  sequencing  of  procedures. 

5.  Favorable  Outcome 

A  hazrep  with  a  favorable  outcome  is  one  in  which  the  correct  decisions  are  made 
for  a  given  scenario  and  the  actions  taken  are  correct  for  the  malfunction.  The  second  half 
of  this  rule  is  in  place  for  two  reasons.  First,  to  prevent  equipment  malfunctions  fi'om 
going  undocumented  and  second,  to  note  the  improper  performance  of  procedures  after 
correctly  diagnosing  the  situation.  For  instance,  a  three  engine  landing  for  a  fire  warning 
is  the  correct  decision  for  this  malfunction.  However,  if  the  fire  warning  is  due  to  a  faulty 
fire  warning  element,  then  an  unnecessary  three  engine  landing  has  been  performed.  The 
crew  made  the  right  decision  based  on  the  information  presented,  but  this  is  not  a 
favorable  outcome.  Similarly,  the  pilot  that  elects  to  secure  an  engine  for  an  unknovra 
fluid  leak  (assuming  worst  case  scenario)  may  have  made  the  correct  decision  if  the  fluid 
was  indeed  a  fuel  leak,  but  not  if  it  turns  out  to  be  a  slight  oil  leak  that  is  within  limits. 
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These  unfavorable  outcomes  are  not  a  reflection  on  the  pilot,  but  rather  on  the  aircraft 
detection  and  warning  systems.  The  unfavorable  outcome  for  the  crew  that  correctly 
diagnoses  the  situation  but  performs  the  wrong  procedures  is  obvious  and  needs  no  further 
explanation. 

6.  Explanations  for  Outcomes 

The  author's  best  attempt  is  made  to  assign  an  explanation  t&  those  hazreps  where 
an  unfavorable  outcome  exists,  judgment  has  been  used,  or  a  favorable  outcome  is 
achieved  and  the  cause  is  readily  apparent.  For  the  majority  of  hazreps  in  which  a 
favorable  outcome  exists  (especially  those  in  the  simple  procedural  rule-based  category) 
there  is  no  assignment  of  correct  decision,  as  this  is  implied  by  the  lack  of  an  unfavorable 
assignment.  Table  3-6  lists  the  explanations  used  and  the  definitions  adhered  to  for  their 
assignment  to  a  hazrep. 

D.  SAMPLE  HAZREP  ANALYSES 

Appendix  A  presents  an  analysis  of  three  hazreps  based  on  the  rules  and  criteria 
outlined  in  this  chapter.  Actual  sample  "EVIDENCE  AND  ANALYSIS,"  "REMARKS" 
and  "CO  COMMENTS"  paragraphs  are  quoted  for  the  reader. 
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Favorable  outcome 
explanations 

Behavior  exhibited  /  Substantiating  criteria 

Correct  decision 

Action  is  appropriate  for  the  situation;  Decision  does  not 
involve  expertise  or  great  thought,  but  an  incorrect  choice 
is  also  available;  lucky  choice  made 

Excellent  SA 

Pilot  has  outstanding  grasp  and  knowledge  of  the  situation; 

Easily  sorts  through  ambiguous  cues  to  find  groimd  truth; 

Reality  apd  pilot's  perception  are  nearly  identical 

Expert  behavior 

Prior  knowledge  of  similar  situation  used;  judgment  used 
to  sort  through  conflicting  data;  past  experiences  used 
to  assimilate  situation;  troubleshooting  used  that  is 
bQ^ond  textbook;  creative  solutions  used  on  unusual  or 
unique  malfunctions 

Unfavorable  outcome 
explanations 

Behavior  exhibited  /  Substantiating  criteria 

Equipment  malfunction 

System  indicators  do  not  corroborate  actual  malfunction 
or  indicate  false  alann/erroneous  data 

Wrong  decision 

Pilot  makes  incorrect  choice  based  on  cues;  bad  luck 

Improper  procedures 

Wrong  choice  of  procedures  for  situation;  correct 
procedures  chosen,  but  performed  incorrectly  or  out  of 

Order 

Lack  of  SA 

Pilot  does  not  grasp  situation;  reality  and  perception 
differ;  correct  decision  made  for  wrong  situation;  tunnel 
vision  exhibited;  pilot’s  dominant  alternative  masks 
correct  one;  ramifications  of  actions  not  considered 

Lack  of  knowledge 

Insufficient  knowledge  to  decipher  cues;  ramifications 
of  actions  unknown;  lack  of  systems  knowledge 
to  properly  troubleshoot  situation 

No  NATOPS  guidance 

Malfimction  not  covered  in  NATOPS;  no  guidance 

Available 

Table  3-6  Explanations  for  Outcome 
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IV.  RESULTS  AND  DISCUSSION 


A  total  of  438  hazreps  covering  years  1996-1999  were  reviewed  and  analyzed  (see 
Appendix  B  for  complete  list  of  hazreps).  This  chapter  presents  the  results  of  the  analysis 
and  develops  a  model  to  illustrate  the  decision  making  process  of  P-3  pilots  as  revealed  in 
this  study.  The  chapter's  first  section  presents  the  results  of  categorizing  the  decision 
events  according  to  the  class  of  rule-based,  knowledge-based  or  skill-based  behaviors,  and 
the  second  section  presents  the  results  of  analysis  of  the  hazreps  in  terms  of  the  major 
contributing  factors  for  both  favorable  and  unfavorable  outcomes.  The  results  are  quite 
extensive  and  an  enormous  amount  of  information  is  presented  (Appendix  C  contains  full 
summary  table).  However,  a  majority  of  the  results  contained  in  the  accompanying  tables 
may  be  of  more  use  for  fiiture  studies  and  P-3  specific  evaluations.  Therefore,  to  maintain 
the  focus  on  decision  making,  two  major  goals  of  this  chapter  are  presented: 

-Identify  those  situations  that  are  best  served  using  an  Analytical  decision  making 
strategy  and  those  where  a  Naturalistic  decision  making  strategy  is  more  applicable; 

-Identify  and  explain  the  major  cause  of  unfavorable  outcomes  in  each  of  the  rule- 
based,  knowledge-based  and  skill-based  classes. 

Accomplishing  these  goals  vdll  provide  a  basis  for  identifying  requirements  to 
better  train  and  equip  decision  makers  in  an  aviation  environment. 

A.  CLASS  OF  DECISION  BEHAVIOR 

The  hazreps  were  categorized  according  to  the  class  of  decision  behavior  (using 
the  characteristics  of  decision  processes  identified  by  Rasmussen:  knowledge-based,  rule- 
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based  or  skill-based  behavior),  type  of  aircraft  system  that  was  most  responsible  for  the 
submission  of  the  hazrep  and  the  category  that  most  accurately  describes  the  situation 
encountered  in  the  hazrep.  Table  4-1  presents  the  percent  of  hazreps  for  each  category 
based  on  the  type  of  decision  making  behavior/environment. 

1.  Rule-based  Decisions 

Rule-based  overall.  The  greatest  number  (78%)  of  hazrepF  were  categorized  as 
involving  rule-based  behaviors.  This  high  percentage  was  expected  and  is  attributable  to  a 
sound  Safety/NATOPS  program  and  years  of  observation  and  experience.  The  intent  of 
the  hazrep  is  to  reduce  future  hazards  by  identifying  them  and  finding  and  incorporating 
into  the  NATOPS  publications  a  subsequent  "cure"  to  prevent  their  recurrence.  One  of 
the  ways  the  Safety/NATOPS  program  accomplishes  this  task  is  by  providing  rules, 
procedures  and  checklists  for  a  crew  to  follow  in  the  event  of  a  malfunction.  The 
NATOPS  program  is  under  constant  review;  changes  to  the  flight  manual  are  submitted 
continuously  and  a  conference  held  annually  to  review  these  submissions  for  additions  to 
the  manual.  The  NATOPS  manual  has  grown  considerably  over  the  years  as  new  hazards 
are  identified  (and  procedures  modified  to  best  handle  a  situation)  and  is  often  referred  to 
as  a  manual  that  is  "written  in  blood":  unfortunately,  the  blood  of  previous  aviators  that 
had  the  misfortune  of  not  having  a  checklist  or  procedure  to  guide  them.  Ideally,  the  rule- 
based  category  would  contain  100%  of  the  hazreps  and  all  malfunctions  would  have  a 
corresponding  procedure  or  textbook  solution. 

The  rule-based  class  was  divided  into  two  subclasses,  based  on  whether  or  not 
judgment  was  applied  when  deciding  on  a  course  of  action.  These  are:  rule-based  simple 
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and  rule-based  judgment.  The  judgment  subclass  contains  those  hazreps  where  pilot 
discretion  was  used  prior  to  deciding  on  a  course  of  action.  However,  the  prescribed 


Class 

% 

Type 

% 

Category 

% 

Airframe 

21 

Ambiguous  indications  (single  COA) 

5 

S 

Electrical 

11 

Compound  factors  (procedural) 

2 

i 

Engine 

39 

Multiple  malfunctions  (procedural) 

4 

m 

90 

Instrument 

0 

Simple  procedural 

85 

U 

NMAC 

0 

Unique;  no  NATOPS  guidance 

2 

u 

Other 

2 

Unusual  malfunction  " 

2 

Press 

3 

Rule 

■ 

Prop 

24 

78% 

j 

Airframe 

18 

Ambiguous  indications  (single  COA) 

6 

u 

Electrical 

18 

Compound  factors  (procedural) 

9 

d 

Engine 

24 

Multiple  malfunctions  (procedural) 

0 

g 

10 

Instrument 

0 

Simple  procedural 

85 

m 

NMAC 

0 

Unique;  no  NATOPS  guidance 

0 

e 

Other 

3 

Unusual  malfunction 

0 

n 

Press 

3 

t 

Prop 

34 

Airframe 

10 

Ambiguous  indications 

KOI 

Electrical 

19 

Compound  factors 

mm 

Knowledge 

Engine 

36 

Multiple  malfunctions 

13 

18% 

Instrument 

6 

Unique;  no  NATOPS  guidance 

6 

NMAC 

9 

Unusual  malfunction 

10 

Other 

2 

Press 

5 

Prop 

13 

Airframe 

0 

Improper  procedures 

56 

Electrical 

0 

Situational  awareness 

22 

Skill 

Engine 

33 

Survival 

22 

4% 

Instrument 

6 

NMAC 

28 

Other 

16 

Press 

11 

Prop 

6 

Table  4-1  Results  by  Class  of  Decision  Behavior,  Type  of  Aircraft  System  and  Category  Describing 

the  Situation 


course  of  action  was  still  relatively  straightforward  and  had  a  clear  procedure  identified  to 
handle  the  malfunction,  thus  it  remained  in  the  overall  rule-based  class. 
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Rule-based  simple.  Ninety  percent  of  hazreps  contained  in  the  rule-based 
class  were  in  the  "simple"  subclass  (this  group  encompasses  70%  of  the  total  sample). 
The  majority  of  tasks  in  the  aviation  domain  are  procedural  in  nature.  Many  years  and 
numerous  resources  have  gone  into  developing  checklists  and  procedures  for  situations 
that  have  occurred  or  can  be  expected  to  occur.  Since  these  checklists  prescribe  the 
appropriate  courses  of  action  for  specified  situations,  in  many  instances,  the  task  of  the 
decision  maker  is  not  to  generate  and  evaluate  a  course  of  action,  but  rather  to  accurately 
assess  the  situation  and  pick  the  appropriate  procedure.  As  mentioned  previously,  ideally 
all  situations  encountered  would  fall  into  this  "simple"  rule-based  class  and  aeronautical 
decision  making  would  be  reduced  to  merely  picking  the  correct  checklist  or  procedure  to 
apply  for  any  given  situation. 

Rule-based  with  judgment.  Ten  percent  of  the  rule-based  class  fall  in  the 
rule-based  with  judgment  subclass  (8%  of  the  total  sample).  This  number  is  considerably 
smaller  than  the  rule-based  simple  category  and  indicates  that  some  rule-based  decisions 
do  require  discretion  or  judgment,  even  though  a  procedure  exists  to  handle  the 
malfimction.  Troubleshooting  unknown  vibrations  or  deciding  on  the  timing  of 
procedures  in  a  complex  scenario  are  examples  of  the  types  of  hazreps  contained  in  this 
subclass. 


a)  Cognitive  Decision  Strategy  Used 

Although  the  pilots  in  the  rule-based  situations  do  not  have  to  generate  and 
evaluate  several  courses  of  action  before  making  a  decision  to  act,  they  still  utilize  an 
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analytical  strategy  during  their  decision  making  process.  They  are  required  to  choose  the 
proper  checklist  or  procedure  from  a  myriad  of  possibilities  contained  in  the  NATOPS 
manual  and  must  use  some  sort  of  structured  or  logical  analysis  to  do  so.  Much  of  this  is 
learned  through  training  and  rote  memorization  of  stimulus-response  scenarios  and 
requires  little  in  the  way  of  judgment  or  experience.  Almost  all  of  the  hazreps  in  the  rule- 
based  category  involve  situations  that  can  be  handled  by  use  of  an  analytical  decision 
making  strategy.  Analytical  decision  making  situations  are  characterized  in  the  following 
way: 

-A  choice  among  clearly  defined  options  exists; 

-Complete  and  reliable  information  is  available; 

-An  optimal  decision  is  desired; 

-Decisions  must  be  justified. 

However,  the  ability  to  correctly  assess  a  situation  and  choose  the  proper 
procedure  may  use  cognitive  skills  that  are  based  on  experience  and  pattern  recognition, 
thus  exhibiting  characteristics  of  the  naturalistic  decision  making  process.  Further,  some 
situations  do  require  judgment  (10%  of  the  rule-based  class  in  this  study)  and  the  pilot  is 
more  apt  to  use  experience  and  knowledge  of  previous  situations  to  assist  in  the  decision 
regarding  which  procedure  to  invoke.  This  use  of  experience  is  characteristic  of  the 
pattern  matching  found  in  the  recognition-primed  decision  strategy  (a  strategy  that  is  part 
of  the  NDM  methodology).  Decisions  categorized  as  using  a  rule-based  behavior  cannot, 
therefore,  be  characterized  as  relying  solely  on  an  analytical  decision  making  strategy. 
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2.  Knowledge-based  Decisions 

Knowledge-based.  Eighteen  percent  of  the  hazreps  analyzed  were  categorized  as 
involving  knowledge-based  behavior.  These  situations  involve  complex  decisions  where 
experience  and  judgment  are  critical.  The  problems  are  such  that  determination  of  a  clear- 
cut  situation  assessment  can  be  difficult  or  the  situations  may  be  compounded  by 
ambiguous  information  or  multiple  malfunctions.  Judgment  is  required  in  100%  of  these 
scenarios.  Pilots  faced  with  this  class  of  decision  require  vast  experience  and  years  of 
training  to  properly  assess  the  situation  and  arrive  at  appropriate  courses  of  action. 
Therefore,  situation  assessment  becomes  at  least  as  important  a  step  as  selecting  from  a 
checklist  or  implementing  a  procedure.  A  critical  component  of  performing  situation 
assessment  for  these  situations  may  be  to  know  when  a  situation  is  unusual  and  does  not 
have  a  corresponding  procedure.  This  requires  that  the  decision  maker  possess  a  wealth 
of  knowledge  concerning  typical  malfunctions  and  scenarios,  and  that  he  be  able  to  relate 
these  to  the  current  situation.  Rule-based  procedures  may  eventually  be  used  in  these 
situations,  but  the  process  used  to  decide  on  which  procedure  to  use  requires  pattern 
matching,  mental  simulation  and  intuition. 

a)  Cognitive  Decision  Strategy  Used 

Experience  and  judgment  are  critical  when  dealing  with  situations  that  are 
ambiguous  in  nature  or  are  compounded  by  multiple  malfunctions.  As  the  situations 
encountered  increase  in  complexity  the  decision  maker  relies  on  familiarity  developed 
through -experience  gained  as  a  result  of  having  dealt  with  similar  t5q)es  of  previous  events. 
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This  experience  base  also  enables  the  decision  maker  to  engage  in  mental  simulation  to 
attempt  to  gain  situational  awareness  and  "role-play"  how  a  particular  course  of  action 
might  play  out  before  implementing  it.  Both  Klein's  RPD  and  Noble's  Situation 
Assessment  models  describe  the  behaviors  exhibited  by  the  decision  makers  in  these 
complex  decision  making  situations.  Using  past  experience  to  recognize  a  familiar 
situation,  mental  simulation  and  role-play  of  future  events  typify  recognition-primed 
decision  making  and  are  the  core  of  knowledge-based  decision  methods.  Almost  all  of  the 
situations  encountered  in  these  hazreps  fit  the  conditions  that  describe  recognition-primed 
decision  (a  strategy  of  the  NDM  methodology); 

-There  is  a  high  degree  of  uncertainty; 

-There  is  ambiguous  or  changing  information; 

-Innovative  or  creative  thought  is  required; 

-Decision  makers  are  knowledgeable  and  experienced. 

It  must  be  noted  that  decisions  made  in  the  knowledge-based  class  are  under  the  same 
scrutiny  as  those  in  the  rule-based  class  and  the  decision  maker  must  often  choose  the 
optimal  course  of  action.  Based  on  this  requirement,  one  could  argue  that  Analytical 
decision  making  is  best  suited  for  knowledge-based  situations.  Actually,  analytical 
behaviors  are  invoked  in  deciding  on  a  course  of  action  once  the  decision  maker  clarifies 
the  situation  and  begins  to  evaluate  possible  courses  of  action.  The  author  believes  that 
this  aspect  of  decision  making  (i.e.  evaluating  courses  of  action)  is  better  suited  to  the 
structured  and  evaluative  approach  of  analytical  decision  making.  However,  the  use  of  the 
recognition-primed  cognitive  strategy  (pattern  matching,  mental  simulation,  expert 
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situation  assessment)  is  also  critical  to  properly  execute  the  process  in  its  entirety. 
Therefore,  as  is  the  case  with  the  rule-based  class,  knowledge-based  situations  require 
cognitive  strategies  from  both  the  analytical  and  naturalistic  methodologies. 

3.  Skill-based  Decisions 

Skill-based.  Four  percent  of  the  hazreps  analyzed  were  categorized  as  involving 
skill-based  behaviors.  These  situations  requiring  skill-based  behaviar  dealt  primarily  with 
preflight  errors  and  the  results  associated  with  the  improper  performance  of  procedures. 
Since  the  hazreps  in  these  instances  were  usually  a  result  of  previous  behavior,  the 
decision  making  process  cannot  be  characterized  due  to  a  lack  of  information  in  the 
hazrep.  The  event  described  in  the  hazrep  was  an  explanation  for  the  handling  of  the 
resultant  malfunction  and  did  not  address  behaviors  that  caused  the  performance  of 
improper  procedures  in  the  first  place.  Based  on  the  experience  and  expertise  of  the 
author  (often  in  corroboration  with  remarks  made  by  the  Commanding  Officer  in  the 
hazrep)  it  is  apparent  that  most  of  the  errors  were  caused  by  a  lack  of  situational 
awareness.  The  skill-based  class  is  included  in  the  study  to  illustrate  the  ensuing  problems 
that  can  occur  with  an  error  in  skill-based  behavior,  such  as  when  a  flight  engineer  leaves 
the  oil  cap  unfastened  which  results  in  an  oil  leak  inflight  and  a  subsequent  three  engine 
landing. 

Also  included  in  this  class  are  the  basic  stimulus-response,  or  automatic  reactions, 
to  near  mid-air  collisions.  No  judgment  is  required  as  this  is  an  instantaneous  reaction  to  a 
life-threatening  situation.  This  is  an  example  of  a  learned  response  that  is  so  well  learned 
that  the  resulting  behavior  is  almost  innate  to  the  decision  maker  and  requires  very  httle 


52 


conscious  effort  or  thought.  An  error  in  this  skill-based  behavior  would  obviously  have 
grave  consequences. 

a)  Cognitive  Decision  Strategy  Used 

The  limited  information  contained  in  these  situations  categorized  as 
involving  skill-based  behavior  makes  it  difficult  to  determine  what  type  of  decision  making 
cognitive  strategy  is  being  used  when  preflight  errors  occur.  Based  on  the  characteristics 
of  the  recognition-primed  decision  making  strategy,  the  instantaneous,  or  intuitive, 
reaction  to  an  approaching  aircraft  could  be  considered  a  case  of  very  rapid  pattern 
matching  and  could  be  considered  to  fit  in  this  category. 

4.  Type  of  Aircraft  System 

Engine  and  propeller  malfunctions  were  the  major  system  contributors  across  the 
entire  sample  of  438  hazreps.  This  pair  of  system  contributors  was  expected,  as  the 
engine  and  propeller  systems  are  the  two  most  complex  systems  on  the  aircraft  and  have 
far  more  moving  parts  and  accessories  than  other  aircraft  systems.  Categorizing  by  type 
of  system  contributing  to  the  malfunction  does  not  provide  much  data  in  the  analysis  of 
decision  making,  but  may  provide  valuable  information  for  engineering,  manufacturing  and 
rework  facilities  for  the  P-3,  as  it  highlights  two  relatively  high  failure  systems. 

5.  Category  Describing  Situation 

a)  Rule-based  Decisions 

Simple  procedural  was  the  standout  category  leader  in  describing  the 
situation  encountered  in  the  rule-based  class,  accounting  for  85%  of  the  total.  As 
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mentioned  previously,  this  is  expected,  as  the  intent  of  the  Safety/NATOPS  program  is  to 
provide  a  simple  "by  the  book"  procedure  for  any  encountered  malfunction. 

b)  Knowledge-based  Decisions 

Situations  involving  compound  factors  (38%)  and  ambiguous  indications 
(33%)  combined  to  make  up  71%  of  the  situations  encountered  in  the  knowledge-based 
class.  These  situations  were  not  covered  explicitly  in  the  NATOPS  and  require  the 
decision  maker  to  use  judgment,  expertise  and  experience  to  correctly  assess  the  situation 
and  decide  on  a  course  of  action. 

B.  OUTCOME  AND  EXPLANATIONS  OF  HAZREP  SITUATIONS 

The  results  included  in  the  hazreps  regarding  favorable  and  unfavorable  outcome 
were  quite  informative  and  are  discussed  below.  Table  4-2  presents  the  results  of  the 
analysis.  It  appears  that  the  likelihood  of  a  favorable  outcome  decreases  as  the  complexity 
of  the  problem  situation  increases.  Clearly  interpretable,  unambiguous  and 
uncompounded  situations  are  more  likely  to  produce  a  favorable  outcome,  as  rules  and 
procedures  exist  to  handle  them. 

1.  Rule-based  Decision  Outcomes 

The  rule-based  simple  class  comprised  the  largest  category  and  had  the  highest 
percentage  of  favorable  outcomes  at  92%.  This  was  expected,  as  the  simple  rule-based 
situations  merely  require  a  quick  and  easy  situation  assessment  followed  by  the  selection 
and  application  of  the  corresponding  procedure.  A  high  level  of  expertise  is  not 
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Class 

% 

1  Favorable  outcome 

1  Unfavorable  outcome 

% 

Explanation 

% 

% 

Explanation 

% 

Q 

Equip  malfunction 

72 

O 

i 

Improper  procedures 

12 

m 

90 

92 

Correct  decision 

100 

8 

Lack  of  knowledge 

0 

u 

Expert  behavior 

0 

Lack  of  SA 

8 

1 

No  NATOPS  guidance 

8 

Rule 

■ 

Wrong  decision 

0 

78% 

J 

Equip  malfunction 

0 

u 

d 

Improper  procedures 

50 

9 

10 

76 

Correct  decision 

64 

24 

Lack  of  knowledge 

0 

m 

Expert  behavior 

36 

Lack  of  SA 

12 

c 

n 

No  NATOPS  guidance 

0 

t 

Wrong  decision 

38 

Equip  malfunction 

29 

Improper  procedures 

11 

Knowledge 

65 

Correct  decision 

27 

35 

Lack  of  knowledge 

11 

18% 

Expert  behavior 

73 

Lack  of  SA 

46 

No  NATOPS  guidance 

Wrong  decision 

- 

Equip  malfunction 

0 

Improper  procedures 

29 

Skill 

22 

Excellent  SA 

100 

78 

Lack  of  knowledge 

0 

4% 

Lack  of  SA 

71 

No  NATOPS  guidance 

0 

Wrong  decision 

0 

Table  4-2  Results  by  Outcome  and  Explanation  of  Hazrep  Situations  Encountered 


necessarily  required  and  the  malfunction  can  usually  be  handled  quickly  and  effectively 
through  applying  the  clear  guidance  contained  in  the  NATOPS  flight  manual. 

Only  8%  of  the  rule-based  category  had  unfavorable  outcomes,  and  the  largest 
contributor  for  an  explanation  of  these  unfavorable  outcomes  was  "equipment 
malfunction"  (72%).  This  equipment  malfunction  is  different  from  an  equipment  failure 
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that  is  considered  routine  and  expected  (e.g.  chips  light  or  oil  leak),  and  is  the  result  of  a 
failure  of  a  detecting  or  indicating  system  (e.g.  oil  transmitter  failure  with  oil  system 
operating  4.0,  or  fire  warning  detection  system  that  gives  an  erroneous  indication).  In 
fact,  almost  all  of  the  pilots  in  these  situations  correctly  performed  procedures  for  the 
indications  exhibited,  but  the  outcome  was  still  considered  "unfavorable"  since  the 
malfunction  was  actually  a  false  alarm  and  involved  an  uimecessary  abort. 

The  rule-based  judgment  class  is  relatively  small  and  there  were  fewer  favorable 
outcomes  (76%  favorable)  than  in  the  rule-based  simple  category.  This  is  probably  due  to 
the  increased  complexity  of  the  decisions  when  judgment  is  required.  Judgment  relies  on 
the  expertise,  experience  and  discretion  of  the  pilot  and  is  affected  by  the  situation. 

The  rule-based  judgment  subclass  involves  decisions  that  are  simpler  than  those  in 
the  knowledge  class  and  have  procedures  more  or  less  spelled  out,  but  still  have  room  for 
error  in  performing  situation  assessment  and/or  the  selection  of  appropriate  procedures. 
While  most  of  the  rule-based  situations  merely  require  a  stimulus-response  type  behavior 
(e.g.  "if 'x'  occurs,  execute  'y'"),  there  is  a  chance  that  the  decision  maker  may  misinterpret 
the  stimulus  "x"  or  fail  to  consider  all  the  cues  presented  in  the  situation.  The  correct 
procedure  in  the  decision  maker's  mind  might  actually  be  contrary  to  the  actual  procedure 
required.  An  example  of  this  type  of  incorrect  selection  of  the  appropriate  procedure  is 
the  pilot  who  mistakenly  performs  pitchlocked  propeller  procedures  for  a  propeller  that 
has  not  yet  pitchlocked.  In  his  assessment  of  the  situation,  the  pilot  mistakenly  beheves 
the  propeller  to  have  already  pitchlocked  and  performs  the  procedures  accordingly. 
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"Improper  procedures"  (50%)  was  the  leading  cause  of  unfavorable  outcomes  in 
the  rule-based  judgment  subclass.  It  appears  that  incorrect  situation  assessment  was  the 
major  basis  for  the  performance  of  improper  procedures  and  can  possibly  be  explained  by 
one  of  the  following.  The  decision  maker  incorrectly  assessed  the  situation  and  selected 
procedures  based  on  this  incorrect  assessment  (pitchlocked  propeller  case  above);  he 
assessed  the  situation  correctly  but  performed  the  wrong  procedure;  or  he  was  locked  into 
a  dominant  alternative  and  failed  to  reassess  a  changing  situation,  thus  performed  the 
wrong  procedure. 

2.  Knowledge-based  Decision  Outcomes 

Ambiguous  indications  (33%)  and  compound  factors  (38%)  combined  accounted 
for  71%  of  the  hazreps  (as  shown  in  Table  4-1)  in  the  knowledge-based  class.  The 
percentage  of  favorable  outcomes  was  65%  and  was  due  largely  to  expert  behavior  (73% 
of  the  favorable  outcomes  were  attributed  to  this).  Commanding  Officer's  comments  on 
favorable  hazreps  all  referred  to  the  experience,  sound  judgment  and  professionalism 
exhibited  by  the  pilots  in  the  favorable  outcome  situations. 

Unfavorable  outcomes  (35%)  were  due  primarily  to  a  lack  of  situational  awareness 
(46%  attributed  to  lack  of  SA).  This  clearly  illustrates  the  need  for  experience  and 
expertise  in  the  cockpit.  Many  of  the  hazreps  with  unfavorable  outcomes  contained 
Commanding  Officer's  comments  regarding  the  inexperience  of  the  crew  or  lack  of  soimd 
decision  making  skills.  "Equipment  malfunction"  (29%)  was  the  second  leading  cause  of 
unfavorable  outcomes  and  it  can  be  easily  seen  that  erroneous  indications  (discussed 
previously  under  rule-based  simple  class)  can  complicate  an  already  ambiguous  situation. 
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Again,  this  was  a  system  indicating  problem  and  does  not  imply  wrong-doing  on  the  part 
of  the  pilot. 

3.  Skill-based  Decision  Outcomes 

The  favorable  outcomes  (22%)  were  all  directly  attributable  to  excellent  S  A  on  the 
part  of  the  pilot  in  avoiding  a  mid-air  collision.  The  78%  of  unfavorable  outcomes  were 
attributed  primarily  to  a  lack  of  SA  on  the  person  responsible  for  the  preflight  error  that 
caused  the  hazrep  situation  (refer  to  Class  of  Decision  Making  Behavior  'Skill-based' 
earlier  in  the  chapter).  Table  4-3  summarizes  the  results  discussed  above. 


Class 

%  Favorable 

#1 

Explanation 

% 

Unfavorable 

#1 

Explanation 

Rule-based 

Simple 

92 

Correct 

decision 

8 

Equipment 

malfunction 

Rule-based 

Judgment 

76 

Correct 

decision 

24 

Improper 

procedures 

Knowledge- 

based 

65 

Expert 

behavior 

35 

Lack  of  SA 

Skill-based 

22 

Excellent  SA 

78 

Lack  of  SA 

Table  4-3  Outcome  synopsis  of  explanations  for  favorable  and  unfavorable  situation  outcomes 

C.  MODELLING  DECISION  MAKING 

A  simplified  model  of  decision  making  based  on  the  analysis  conducted  in  this 

study  is  presented  in  Figure  4-1.  It  illustrates  the  inputs  and  outputs  to  memory  as  the 
processes  of  the  OODA  loop  discussed  in  Chapter  H.  As  can  be  seen,  a  decision  maker 
attempts  to  gain  situational  awareness  by  first  processing  the  observed  cues  in  working 
memory.  '  If  an  immediate  response  is  not  readily  apparent,  the  decision  maker  must 


attempt  a  pattern  match  with  templates  of  prior  experience  stored  in  long  term  memory. 
A  decision  is  then  derived  in  working  memory  and  an  appropriate  action  performed. 

Figure  4-2  is  a  more  elaborated  model  of  the  decision  processes  used  by  pilots  for 
the  problem  situations  encountered  in  this  study.  It  is  a  hybrid  model  that  combines 
properties  of  the  OODA  loop,  cognitive  hierarchy,  pattern  matching  model  and 
Rasmussen's  (1983)  decision  behaviors.  The  figure  shows  two  major  processes  working 
concurrently:  the  decision  making  processes  of  the  OODA  loop  (Observe,  Orient,  Decide 
and  Act)  and  the  information  processes  of  the  cognitive  hierarchy  (data  being  transformed 
to  information,  and  then  to  knowledge,  and  finally  to  understanding).  The  boxes  (outlined 
in  bold)  of  the  OODA  loop  represent  major  stages  in  the  processing  of  information,  two  of 
which  are  located  in  working  memory.  Although  working  memory  is  depicted  as  two 
separate  levels,  it  should  be  thought  of  as  one  entity,  as  depicted  in  Figure  4-1. 


Figure  4-1  Simplified  decision  model  depicting  inputs  and  outputs  to  memory 
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As  can  be  seen,  skill-based  and  rule-based  simple  decisions  require  little  effort  from 
working  memory.  As  the  pilot  processes  the  information  and  begins  the  orient  phase,  he 
either  reacts  immediately  to  the  stimulus  (e.g.  near  mid-air  collision  avoidance)  or  readily 
pattern  matches  the  cue  to  a  specific  procedure  or  checklist.  Errors  resulting  in 
unfavorable  outcomes  at  this  stage  are  caused  primarily  from  erroneous  systems 
indications  and  not  human  error.  Extensive  training  and  repetition  of  procedures  produces 
pilots  that  are  quite  effective  at  matching  the  procedure  to  the  malfunction  and  then 
performing  the  procedures  correctly.  Only  when  a  pattern  match  is  not  obtained 
immediately  does  the  decision  maker  have  to  process  the  information  further  and  resort  to 
long-term  memory  for  a  possible  stored  event  that  is  congruent  with  the  situation  at  hand. 

This  venture  into  long-term  memory  uses  cognition  and  learned  aspects  of  the 
domain  (e.g.,  failure  modes,  operating  characteristics,  previous  hazreps,  etc.)  and  results 
in  the  pilot  gaining  a  clearer  situation  assessment  or  "knowledge  picture"  (refer  to 
cognitive  hierarchy  discussion  in  Chapter  II)  of  the  situation.  The  author  believes  that  the 
more  effectively  the  decision  maker  can  perform  pattern  matching  in  working  memory 
(after  retrieving  an  experience  from  long-term  memory),  the  more  effectively  the  problem 
situation  can  be  handled.  This  process  is  representative  of  the  expert  behavior  seen  in  the 
knowledge-based  hazreps  and  is  best  performed  by  more  experienced  pilots.  Experienced 
pilots  have  a  much  broader  knowledge  base  to  draw  from  and  can  gain  situational 
awareness  faster. 
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Figure  4-2  Hybrid  decision  model  for  P-3  pilot  hazrep  decision  making  in  problem  situations 


Once  situational  awareness  is  obtained  the  pilot  must  decide  on  a  course  of  action. 
This  process  is  once  again  centered  in  working  memory  and  can  heavily  tax  the  decision 
maker's  attentional  resources  and  reasoning  ability.  The  rule-based  judgment  class  is  less 
demanding  on  working  memory  and  does  not  require  the  greater  understanding  that 
knowledge-based  decisions  do.  If  previous  knowledge,  past  experience  or  domain  specific 
knowledge  have  not  provided  a  pattern  match  (or  similar  match)  then  the  pilot  must 
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integrate  cues  and  generate  an  original  solution  to  the  problem.  This  creative  process  may 
further  tax  the  attentional  resources  of  working  memory. 

The  "loss  of  SA"  cloud  shown  in  the  upper  left  comer  of  the  figure  expands  as 
stress  and  time-pressure  increase,  effectively  reducing  the  capability  of  working  memory. 
Losing  situational  awareness  or  performing  improper  procedures  for  the  malfunction  at 
hand  most  often  causes  errors  in  this  region.  Again,  experience  may  be  the  critical 
component  to  alleviating  this  loss  of  SA  as  uncertainty  and  stress  increase. 

A  caution  regarding  experience  must  be  mentioned  at  this  point.  It  is  conceivable 
that  a  pilot  with  extensive  past  experience  could  "fixate"  on  an  alternative  that  is  not 
appropriate  for  the  situation.  As  working  memory  capacity  is  taxed  and  the  SA  cloud 
grows,  one  alternative  may  dominate  the  decision  maker's  judgment  so  that  it  is  seen  as 
the  only  viable  alternative.  Under  stress  (e.g.  ambiguous  information,  time-pressure,  etc.) 
decision  makers  are  prone  to  develop  a  type  of  "cognitive  tunnel  vision." 

Not  explicitly  shown  in  the  figure  is  a  feedback  loop  that  occurs  after  an  action  has 
been  performed.  This  feedback  presents  itself  as  data  entering  the  observation  phase. 
Thus,  the  cycle  is  an  iterative  loop  that  allows  for  flexibility  and  change. 
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V.  CONCLUSIONS  AND  RECOMMENDATIONS 
This  thesis  examined  decision  making  in  a  P-3  flight  station  during  malfunction 
situations.  An  analysis  of  438  hazreps  was  conducted  to  distinguish  different  kinds  of 
decision  situations  encountered  and  whether  a  particular  decision  strategy  from  either  the 
Analytical  or  Naturalistic  decision  making  methodologies  is  more  appropriate  for  a  given 
situation.  Situation  outcomes  were  analyzed  to  pinpoint  the  cause  of  unfavorable  results 
in  an  attempt  to  prevent  their  reoccurrence  in  the  future. 

It  is  clear  from  the  results  that  the  majority  of  situations  analyzed  involve  rule- 
based  decisions  and  can  be  performed  with  relative  ease  by  matching  a  procedure  to  a 
malfunction.  More  complex  decisions  are  required  when  the  situation  is  clouded  with 
ambiguity  or  compounded  by  additional  factors.  These  knowledge-based  decisions 
require  judgment  and  experience  on  the  part  of  the  decision  maker.  Experience  appears  to 
be  the  critical  component  in  gaining  and  maintaining  situational  awareness  and  making 
effective  decisions.  Strategies  from  both  the  Naturalistic  and  Analytical  decision  making 
methodologies  have  a  place  in  the  decision  making  processes  of  pilots  in  this  study  and 
must  be  considered  when  training  new  personnel.  Table  5-1  summarizes  the  key  aspects 
of  the  study  and  provides  the  following  observations: 

1)  Rule-based  decisions  comprise  the  vast  majority  of  situations  encountered  in 
the  aircraft; 

2)  Knowledge-based  decisions  require  experience,  intuition,  previous  knowledge 
and  expert  behavior; 
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3)  The  probability  of  an  unfavorable  outcome  increases  as  the  complexity  of  the 
environment  increases; 

4)  A  need  exists  for  better  detecting  and  indicating  systems  for  malfunctions; 

5)  Situational  awareness  is  key  to  preventing  unfavorable  outcomes; 

6)  Recognition-primed  and  analytical  decision  making  strategies  are  both  required 
for  effective  decisions. 


Class 

% 

Mam  decision  making 
method 

Explanation 

Rule  simple 

70 

Analytical 

8 

Equipment 

malfunction 

Rule  judgment 

8 

Analytical  w/  NDM 

24 

Knowledge 

18 

Naturalistic  w/  ADM 

35 

Lack  of  SA 

Skill 

B 

Naturalistic  (NMAC) 

78 

Lack  of  SA 

Table  5-1  Type  of  decision  making  strategy  associated  with  class  of  decision  type  and  explanations 

for  unfavorable  outcome 

A.  RECOMMENDATIONS 

Recognizing  the  need  for  the  use  of  strategies  from  both  the  Analytical  and 
Naturalistic  Decision  Making  methodologies  is  critical  if  the  Navy  wants  to  reduce  the 
number  of  unfavorable  outcomes  in  hazardous  situations  encountered  in  Naval  Aviation. 
The  recommendations  below  are  intended  to: 

-  Increase  situational  awareness  among  pilots  in  order  to  reduce  the  probability  of 
errors  associated  with  improper  situation  assessment; 

-  Alleviate  erroneous  aircraft  indicating  systems  in  order  to  reduce  ambiguous 
situations  and  unnecessary  aborts. 
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1.  Train  Pilots  to  Become  Experts 

The  military  conducts  ab  initio  training;  that  is,  it  transforms  complete  novices  into 
minimally  proficient  and  safe  pilots  (Kaempf  &  Orasanu,  1997).  The  basic  flight  training 
takes  approximately  one  year,  followed  by  specific  in-model  training  at  a  Fleet 
Replacement  Squadron  and  an  upgrade  syllabus  in  an  operational  squadron.  A  typical  P-3 
Plane  Commander  requires  approximately  18-24  months  to  completeThe  upgrade  syllabus. 
After  designation,  a  fiilly  upgraded  pilot  has  little  in  the  way  of  training  requirements  other 
than  annual  refresher  training.  This  type  of  training  produces  a  vast  number  of  competent 
pilots  that  have  very  little  experience  to  draw  from.  In  addition,  a  pilot's  tour  (especially 
Navy  pilots)  not  only  involves  flying  but  also  performance  of  a  ground  job  and  associated 
military  duties  and  watches.  Flying  frequently  becomes  a  secondary  task.  The  result  is  a 
pilot  who  is  proficient  at  handling  the  routine  rule-based  decisions  but  does  not  necessarily 
have  the  expertise  to  handle  the  more  complex  knowledge-based  decisions. 

The  first  prerequisite  for  improving  situational  awareness  and  the  skills  to  perform 
the  pattern-recognition  strategy  is  to  increase  expertise.  The  more  comprehensive  a  pilot's 
knowledge,  the  easier  it  is  to  find  a  pattern  match  in  long  term  memory  to  deal  with  a 
hazardous  situation.  Pilots  should  practice  decision  making  skills  in  time-limited  and 
stressful  situations  on  a  daily  basis.  Training  must  not  be  confined  to  staff  courses,  aimual 
refresher  training  or  postgraduate  education,  but  be  instituted  on  a  regular  basis  in 
operational  squadrons.  Decision  training  should  be  incorporated  into  flight  training  from 
the  onset  and  continued  throughout  a  pilot's  career. 
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Personal  experience  is  the  best  way  to  acquire  expertise,  but  this  is  not  always 
possible.  The  use  of  simulators,  decision  making  exercises  and  "hangar  flying"  must  be 
mandated.  Simulation  provides  an  excellent  method  of  training  on  a  large  number  of 
challenging  and  realistic  scenarios  in  a  short  period  of  time.  Situations  can  be 
programmed  in  simulators  to  replicate  actual  aviation  hazards  that  have  occurred  in  the 
past.  Simulation  also  can  present  the  decision  maker  with  the  problem  of  analyzing 
ambiguous  or  complex  situations  without  the  fear  of  failure  (e.g.,  loss  of  life  or  aircraft). 
Using  historical  analyses  of  hazardous  situations  and  real  time  feedback  can  vastly 
improve  the  decision  making  processes  of  pilots. 

This  is  not  to  say  that  simulators  are  not  currently  used  in  the  operational  fleet;  in 
fact,  they  are  an  integral  part  of  all  the  training  tracks  that  the  author  has  been  associated 
with.  However,  once  a  pilot  completes  his/her  prescribed  training  and  becomes  a  Plane 
Commander;  there  is  a  dramatic  decrease  in  mandatory  simulator  training  required.  The 
time  that  is  required  in  simulation  is  usually  spent  conducting  instrument  proficiency 
training  rather  than  improving  decision  making  skills. 

Group  decision  making  exercises  and  pilot  forums  also  provide  an  excellent  means 
of  training  junior  (qualified  and  unqualified)  pilots.  Revievdng  previous  hazardous 
situations  and  discussing  the  way  in  which  the  they  were  handled  by  other  decision  makers 
can  build  a  pilot's  experience  base.  Further,  a  group  discussion  evokes  other's  opinions  on 
alternative  courses  of  action  and  the  decision  strategy  used  to  arrive  at  that  course  of 
action.  These  opportunities  provide  an  excellent  resource  for  transferring  experience  and 
expertise  directly  from  one  pilot  to  another. 
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2.  Utilize  Existing  Expertise 

A  Naval  aviator  frequently  is  given  an  enormous  amoimt  of  responsibility  while 
still  very  junior.  For  example,  the  author  was  a  P-3  Plane  Commander  as  a  LTJG  and  a 
Mission  Commander  and  Instructor  Pilot  as  a  junior  LT.  While  this  amount  of 
responsibility  is  sought  after  and  a  tribute  to  one's  abilities,  it  may  not  be  in  the  best 
interest  of  properly  training  junior  decision  makers.  A  better  approach  may  be  to  pair 
novice  aviators  with  seasoned  and  experienced  pilots  for  a  longer  period  of  time  after  the 
novice  is  qualified.  This  gives  the  newly  designated  Plane  Commander  the  opportunity  to 
hone  the  decision  making  skills  necessary  for  making  the  more  complex  knowledge-based 
decisions.  Furthermore,  this  also  provides  the  experienced  pilot  a  fresh  "rule-based 
philosophy"  from  the  junior  decision  maker  that  is  not  clouded  by  bias,  prejudice  or  ego. 
This  naive,  "book-learned"  perspective  from  the  junior  aviator  could  alleviate  unfavorable 
outcomes  in  hazardous  situations  caused  by  experienced  pilots  who  persist  in 
implementing  undesirable  (dominated)  alternatives  when  better  choices  are  available. 

3.  Incorporate  Naturalistic  Decision  Making  Strategies  in  the  Training  of 

Pilots 

This  study  exemplifies  the  need  for  training  decision  makers  in  strategies  from  both 
the  Analytical  and  Naturalistic  Decision  Making  methodologies.  However,  the  application 
of  NDM  research  in  aviation  is  still  in  its  infancy.  To  the  best  of  the  author's  knowledge, 
there  are  no  courses  presently  taught  in  the  P-3  aviation  community  that  address  NDM. 
Naturalistic  decision  making  strategies  are  most  applicable  when  encountering  dynamic 
and  continually  changing  conditions,  real  time  reactions  to  these  changes,  ill  defined  tasks. 
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time  pressure,  significant  personal  consequences  for  mistakes,  and  experienced  decision 
makers.  As  this  study  illustrates,  these  conditions  prevailed  in  at  least  18%  of  hazardous 
situations;  i.e.,  when  knowledge-based  decisions  were  required.  Using  the  recognition- 
primed  strategy,  experienced  decision  makers  assessed  the  situation  and  recognized 
familiar  patterns.  In  fact,  the  recognition-primed  strategy  was  probably  used  in  virtually 
every  situation  encountered  for  the  initial  situation  assessment.  IHs  vital  that  pilots  be 
aware  and  trained  in  a  decision  making  methodology  that  may  help  them  make  a  life 
saving  decision  in  the  future. 

4.  Improve  Reporting  Procedures  to  Include  Decision  Making  Strategies 

UtUized 

While  conducting  this  analysis  of  P-3  hazreps  the  author  discovered  that  the  hazrep 
message  was  lacking  in  information  that  would  have  proved  helpful  in  deciphering  the 
decision  making  process  used.  The  basis  for  the  pilot's  situation  assessment  (e.g., 
NATOPS,  past  experience,  inferred  fi-om  historical  knowledge,  etc.),  discussions 
regarding  how  a  decision  was  arrived  at,  alternatives  considered,  and  whether  time 
constraints  played  a  vital  role  in  the  decision  process  could  have  shed  more  light  on  the 
subject.  A  "decision  strategy"  section  of  the  hazrep  could  provide  pilots  a  valuable 
discussion  item  for  their  group  decision  making  exercises  and  "hangar  flying"  sessions. 
The  practice  of  providing  complete  and  forthright  information  in  the  hazrep  must  not  only 
be  encouraged,  but  enforced,  to  ensure  that  all  the  actions  of  the  decision  maker,  right  or 
wrong,  are  clearly  explained  and  documented.  In  this  way,  junior  decision  makers  can 
evaluate  the  situations  and  actions  for  themselves  and  build  their  own  experience  base. 
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5.  Utilize  a  Decision  Support  System 

Decision  support  systems  can  be  utilized  to  provide  the  decision  maker  with  timely 
and  accurate  information  to  aid  the  decision  making  process.  Recall  that  the  largest 
contributor  for  an  explanation  of  unfavorable  outcomes  in  the  rule-based  simple  class  was 
"equipment  malfunction"  (72%).  This  equipment  malfunction  is  different  from  an 
equipment  failure  that  is  considered  routine  and  expected  (e.g.  chipriight  or  oil  leak)  and 
is  the  result  of  a  failure  of  a  detecting  or  indicating  system.  A  decision  support  system 
could  not  only  provide  valuable  insight  to  the  decision  maker  to  prevent  these  unnecessary 
aborts,  but  also  could  aid  the  pilot  with  situation  assessment. 

One  such  system  currently  under  development  by  the  USAF  and  NASA  is  Hazard 
Monitor  (HM).  Hazard  Monitor  is  a  knowledge-based  aid  designed  to  reduce  the  rate  of 
preventable  accidents  regardless  of  the  source  of  the  problem  (i.e.,  human,  machine,  or 
external  environment).  The  idea  for  the  system  stems  from  the  fact  that  in  complex 
situations,  hazards  occur  despite  improvements  in  system  design  and  advances  in  human- 
computer  interaction.  HM  aids  these  situations  by  enhancing  the  problem  recognition  and 
identification  process  so  that  operators  recognize  deteriorating  situations  in  time  to  avoid 
adverse  consequences.  (Emst-Fortin  et  al.,  1997) 

6.  Replace  Faulty  Aircraft  Indicating  Systems 

In  the  absence  of  a  decision  support  system,  simply  modifying  the  aircraft  to 
reduce  the  number  of  false  indications  would  greatly  reduce  the  number  of  hazardous 
situations  encountered.  Faulty  fire  warning  elements  and  oil  pressure/quantity  transmitters 
were  the  high  failure  items  and  accounted  for  a  majority  of  the  "equipment  malfunction" 
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category.  Failures  involving  fire  warnings  and/or  the  oil  system  usually  result  in  a  three 
engine  landing.  While  three  engine  landings  are  well  trained  for  and  often  considered 
"routine"  almost  all  Commanding  Officers  will  tell  you  "four  engines  are  always  better 
than  three."  The  cost  of  replacing  faulty  indicating  systems  would  most  likely  be  far 
outweighed  by  the  benefits  realized. 
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APPENDIX  A.  SAMPLE  HAZREP  ANALYSES 
Appendix  A  contains  selected  sections  from  three  sample  hazreps  from  each  of  the 

classes  (knowledge-based,  rule-based  and  skill-based)  with  an  accompanying  analysis 

based  on  the  rules  and  criteria  outlined  in  Chapter  HI. 


71 


KNOWLEDGE-BASED  DECISION  EXAMPLE 


EVIDENCE  AND  ANALYSIS.  Immediately  following  takeoff  at  NAS 
TAX  for  a  USW  event,  #1  chips  light  illuminated  with  no  secondaries.  PPC 
requested  downwind,  and  once  established,  shutdown  #1.  The  restart 
checklist  was  completed  through  PCO.  The  following  items  were  then 
discussed:  restarting  #1,  dumping  the  remaining  fuel  in  tank  5,  weather, 
and  overweight  landing.  Two  engine  performance  based  on  gross  weight 
and  OAT  was  determined.  Of  note,  landing  ground  roll  distance  was  not 
discussed.  The  PPC  elected  to  perform  an  overweight  landing  based  on  the 
two  engine  performance  and  the  forecast  of  Thunder  I  conditions  prior  to 
arriving  at  114,000  lbs.  The  three  engine  landing  was  performed  on 
runway  27  (dry,  8000  ft)  at  approximately  120,000  lbs,  with  2000  lbs  of 
fuel  remaining  in  tank  5.  Winds  were  180/6.  Landing  touchdown  speed 
was  142  kts  with  6500  ft  remaining.  Brakes  were  not  utilized  until 
approximately  2500  ft  remaining.  The  aircraft  was  stopped  with 
approximately  200  ft  remaining.  Brake  cooling  was  neither  discussed  nor 
performed.  Taxiing  into  the  parking  spot  and  following  engine  shutdown, 
the  lineman  signaled  hot  brakes  on  port  side  followed  by  a  brake  fire  signal. 
Crew  called  for  assistance  and  egressed  over  starboard  wing. 

REMARKS.  Postflight  discussions  about  the  decision  to  land  heavy 
brought  up  items  for  concern.  1.  Post  flight  determination  of  the  landing 
ground  roll  distance  was  computed  to  be  about  4200  ft  with  moderate 
braking.  This  figure  probably  would  not  have  prevented  the  PPC  fi'om 
making  the  overweight  landing  but  it  should  have  spotlighted  the  need  for 
brakes  and  perhaps  brake  cooling.  2.  The  PPC  used  the  absolute  worst 
case  scenario  to  arrive  at  his  decision  to  land  the  aircraft  at  120,000  lbs;  the 
potential  loss  of  another  engine  and  forecast  convective  weather.  This 
decision  path  reduced  his  options  and  shortened  his  timeline. 

CO  COMMENTS.  The  PPC  decided  that  the  potential  for  another  engine 
malfunction  and  worsening  weather  was  more  hazardous  than  the  risks 
associated  with  an  overweight  three  engine  landing  on  a  hot  Jacksonville 
day.  The  crew  discussed  what  could  happen  if  they  remained  in  the  air  but 
they  failed  to  fully  discuss  what  could  happen  if  they  landed  in  their  present 
configuration.  Overlooked  were  items  including  ground  roll  distance, 
moderate  braking  and  brake  cooling  procedures.  Although  an  "uneventful" 
landing  ensued,  the  brake  fire/hot  brakes  were  only  one  of  the  many 
possible  outcomes  of  this  scenario.  The  more  prudent  course  of  action 
would  have  been  to  continue  overhead  the  field,  prepared  to  relight  the 
feathered  engine  and  bum  down  to  a  lower  weight  prior  to  landing. 
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ANALYSIS  OF  KNOWLEDGE-BASED  HAZREP 


Type:  Engine  (chips  light) 

Class:  Knowledge  (multiple  alternatives  available,  thorough  understanding  of 
environment  required  before  taking  action) 

Category:  Compound  factors  (heavy  aircraft,  weather) 

Judgment  required:  Yes  (PPC  elected  to  land  heavy,  PPC  chose  worst  case 
scenario) 

Favorable  outcome:  No  (resulting  brake  fire  could  have  been  avoided  if  better 
decision  made) 

Explanation:  Lack  of  SA  (PPC  perceived  his  worst  case  scenario  as  the  deciding 
factor;  crew  failed  to  fully  consider  ramifications  of  landing  heavy) 

Comments:  The  question  may  arise,  if  the  outcome  had  been  favorable,  would  the 
explanation  have  been  "expert  behavior?"  The  answer  is  no.  If  the  PPC  had  elected  to 
wait  overhead  the  field  and  commence  the  approach  at  a  lighter  gross  weight,  or  if  he  had 
considered  the  use  of  brakes  and  brake  cooling  procedures,  the  explanation  would  have 
been  "correct  decision."  Full  consideration  of  performance  factors  and  proper  application 
of  NATOPS  procedures  is  not  considered  "expert"  behavior.  Expert  behavior  requires 
going  beyond  normal  procedures  and  uses  experience,  judgment  and  (sometimes)  prior 
knowledge  to  arrive  at  a  decision. 
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RULE-BASED  DECISION  EXAMPLE 


EVIDENCE  AND  ANALYSIS.  On  station  at  2550  ft  AGL,  crew  had  two 
engines  loitered  at  a  gross  weight  of  100,000  lbs.  Crew  thoroughly 
discussed  three  and  two  engine  loiter  considerations  and  determined  single 
engine  rate  of  descent  to  be  420  FPM.  Number  one  engine  had  been  shut 
down  for  5.5  hours  and  #4  engine  for  one  hour  when  FE  called  out  #3 
RPM  flux  with  decreasing  RPM  toward  zero  percent  with  no  audible 
indications.  With  PPC  concurrence,  FE  initiated  restart  on  #4  engine. 
During  restart,  FE  called  #3  oil  pressure  low  light,  master  electrical  power 
light  and  master  pressurization  Ught.  As  FE  called  #4  normal  restart,  #3 
chips  hght  illuminated.  Crew  secured  #3  engine  and  completed  emergency 
shutdown  checklist  through  item  6.  With  power  on  #4  limited  due  to  low 
oil  temp,  the  PPC  initiated  a  500  FPM  rate  of  descent  to  maintain  loiter 
airspeed.  Engine  #1  was  restarted  with  normal  indications.  Crew 
completed  emergency  shutdown  checklist  for  #3  engine  followed  by  restart 
checklist  for  engines  #1  and  #4.  Power  became  available  on  engine  #4 
within  one  minute  of  restart  and  within  three  minutes  for  engine  #1. 
Aircraft  minimum  altitude  1800  AGL.  Crew  initiated  climbout  for  RTB 
and  checked  offstation  while  discussing  considerations  for  a  second  engine 
failure.  Weather  was  updated,  crew  briefed  three  engine  landing 
considerations,  declared  an  emergency,  and  conducted  an  uneventful  three 
engine  landing. 


REMARKS.  Postflight  inspection  revealed  a  cracked  idler  gear  in  #3 
reduction  gearbox  which  resulted  in  loss  of  entire  RGB  accessory  section. 


CO  COMMENTS.  Sohd  crew  coordination  is  required  to  safely  conduct 
two  engine  loiter  operation.  Initial  engine  #3  malfunction  indications  did 
not  require  immediate  action,  so  the  PPC  made  the  decision  to  restart  #4 
first.  By  the  time  additional  indications  required  the  shutdown  of  #3 
engine,  #4  was  on  line.  Although  #4  was  available  if  needed,  with 
sufficient  altitude  to  accept  a  descent  while  waiting  for  engine  oil 
temperature  to  increase,  the  PPC  restarted  #1.  Total  altitude  lost  was  700 
ft,  suggesting  both  engines  were  on  line  in  less  than  2  minutes.  Clear 
understanding  of  the  two  engine  loiter  brief,  sound  decision  making  and 
professional  execution  ensured  safe  recovery  from  a  situational  risk  a  PPC 
accepts  when  he  decides  to  loiter  two  engines. 
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ANALYSIS  OF  RULE-BASED  HAZREP 


Type:  Engine  (RGB,  chips  light) 

Class:  Rule-based  (NATOPS  dictates  action) 

Category:  Compound  factors  (two  engine  loiter,  altitude  and  rate  of  descent 
considerations) 

Judgment  required:  Yes  (PPC  elected  to  restart  #4  prior  to  securing  #3,  held 
emergency  shutdown  checklist  prior  to  completion  to  perform  restart  on  #1) 

Favorable  outcome:  Yes  (correct  decisions  made  and  actions  taken  fit  the 
situation) 

Explanation:  Expert  behavior  (PPC  fiilly  aware  of  status  of  #3  engine,  judgment 
used  to  delay  securing  of  #3  engine  with  indications  of  pending  failure,  prioritization  of 
procedures  required  to  minimize  altitude  loss  and  acceptance  of  slight  loss  of  altitude  to 
maintain  loiter  airspeed) 

Comments:  This  particular  emergency  situation  could  have  easily  ended  in  disaster 
for  another  crew  faced  with  the  same  situation.  A  misinterpretation  of  the  indications  on 
#3  could  have  led  a  crew  to  securing  it  prior  to  initiating  restart  on  #4.  The  aircraft  would 
have  then  been  operating  on  a  single  engine  with  a  rate  of  descent  of 420  FPM.  Further,  if 
a  PPC  disregarded  the  need  to  maintain  loiter  airspeed  during  the  restarts,  he  could  easily 
place  the  aircraft  in  a  stall  as  airspeed  bleeds  off 
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SKILL-BASED  DECISION  EXAMPLE 


EVIDENCE  AND  ANALYSIS.  The  crew  had  just  completed  starting  the 
#2  engine  utilizing  a  huflfer  in  preparation  for  a  functional  check  flight 
(FCF).  After  making  a  quick  scan  of  the  instruments  and  noting  a  normal 
start,  both  the  PPC  and  the  CP  noticed  the  aircraft  was  slowly  moving 
forward.  Both  pilots  quickly  depressed  the  brakes,  which  immediately 
stopped  the  aircraft  after  an  estimated  sbc  feet  of  movement. 
Simultaneously  the  lineman  chocked  the  main  landing  gear,  ensuring  no 
further  movement  of  the  aircraft  was  possible.  Neither  pilot  noticed  if  the 
brake  handle  was  set  prior  to  using  the  brakes  to  stop  the" aircraft.  The 
aircraft  was  secured  and  maintenance  informed.  The  entire  brake  system 
was  checked  with  no  discrepancies  found. 

REMARKS.  After  reviewing  the  scenario,  the  PPC  decided  that  he  must 
not  have  ensured  the  brakes  were  properly  set  during  the  before  start 
checklist.  With  no  specific  recollection  of  any  deviation  fi'om  standard 
procedure,  the  only  explanation  can  be  inattention  during  the  checklist. 

CO  COMMENTS.  This  highly  qualified  crew  was  faced  with  the  challenge 
of  a  short  notice  FCF.  Such  flights  occur  in  the  course  of  normal 
operations  and  are  a  part  of  doing  business.  The  learning  point  is  the  need 
for  schedulers  and  crews  to  identify  this  type  of  added  hazard  in  their  ORM 
process.  Heightened  attention  to  detail  could  have  prevented  this  scenario. 


ANALYSIS  OF  SKILL-BASED  HAZREP 

Type:  Other  (basic  skill  error) 

Class:  Skill-based  (procedure  exists  on  "Before  Start"  checklist  to  set  parking 
brake — a  common  skill) 

Category:  Improper  procedures  (did  not  perform  checklist  item) 

Judgment  required:  No 
Favorable  outcome:  No 

Explanation:  Improper  procedures  (see  category) 

Comments:  None 
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APPENDIX  B.  LIST  OF  HAZREPS 

Appendix  B  contains  the  complete  list  of  hazreps  (438)  analyzed  in  this  study 
(Excel  format).  The  hazreps  are  sorted  by  class  (knowledge-based,  rule-based  and  skill- 
based)  then  by  category  that  best  describes  the  situation  (ambiguous  situation,  compound 
malfunction,  etc.). 
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Subject  Line  _ Type  Class  Category _ Results _ CO  comments _ My  comments _ Judgment  Explain  Unfav 

'  ^  Elected  to  secure  based 

Ambiguous  on  unknown  fluid;  oil  leak 

Oil  leak _ ^ _ Eng _ k  indications  Unnecessary  3  eng  Idg  within  limits  x  Equip  x 


X 


X 
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Oil  leak _ Eng _ k  Indications _ 3  eng  Idg _ except  for  qty  gage 

Ambiguous  OH  press  and  qty  norms 

Oil  leak _ Eng _ k  indications _ 3  eng  idg _ despite  leak _ 

Ambiguous  Fit  station  indications 

Oil  leak _ Eng _ k  indications _ 3  eng  Idg _ normal;  turbine  oil  leak 


Subject  Line  . _ Type  Class  Category _ Results _ CO  comments _ My  comments _ Judgment  Explain  Unfay 

Ambiguous  Elected  to  divert  for  wx 

CPI  failure  without  off  flag _ Jnst _ indications _ Divert  ■ _ and  suitable  approach  x  CD 

Ambiguous  Could  not  determine  type 

#2  EDC  failure  &  leak  Press  k  Indications  3  eng  Idg  of  leak:  secured  enalne  x  CD 
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Never  should  have 

Abort  leads  to  brake  reached  this  point  of 
Prop  underspeed  on  t-off _ Prop  k  [Compound  factors  fire _ extremis _ 


Subject  Line  _ Type  Class  Category _ Results _ |C0  comments _ |My  comments _ |  Judgment  |  Explain  |  Unfav 
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NMAC _ NMAC  k  Compound  factors _ workload;  time  to  respond 

ATC  incorrect  call;  evasive 

NMAC  with  civilian _ NMAC  k  Compound  factors _ req'd _ 

NMAC _ NMAC  k  Compound  factors _ Excellent  SA _ Immediate  evasive  action 

NMAC  with  DC~10 _ NMAC  k  Compound  factors _ Evasive  action  judgment 


Subject  Line  _ Type  Class  Category _ Results _ CO  comments _ My  comments _ Judgment  Explain  Unfav 

Disparity  between  actions 

Bird  strike _ ^ _ Other _ k  Compound  factors  Aborted  t-off _ Sound  crew  coordination  of  PAG  and  PPG  x  Exp 
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Ghips  light  and  oil  leak _ Eng _ k  malfunctions _ 3  eng  Idg _ IMG 

Multiple  Unusual  compound 

Oil  qty  and  PP  light _ [Eng  |  k  | malfunctions  {3  eng  Idg _ [emergency _ Good 


Subject  Line _ Type  Class  Category _ Results _ CO  comments _ My  comments _ Judgment  Explain  Unfav 
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83 


FOUO  '  Elec  r  Simple  procedural  Evacuated  aircraft 
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Subject  Line  _ Type  Class  Category _ Results _ CO  comments _ My  comments _ Judgment  Explain  Unfav 

'  ,  Multiple  Reacted  properly  to  Lightning  strike  after  fire 

Fire  warnirig  _ Eng _ r  malfunctions _ Unnecessary  3  eng  Idg  compound  emergency  warn;  faulty  fire  warning _ Equip  x 

Multiple  •  Some  inaccurate 
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Fire  warning _ Eng  r  Simple  procedural  Unnecessary  3  eng  Idg  _ {Fire  warning  element  | _ |  Equip 


Subject  Line  _ Type  Class  Category _ Results _ CO  comments _ [My  comments _ [  Judgment  |  Explain  |  Unfav 


X 


X 
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EDC  failure _ Press  r  Simple  procedural  3  eng  Idg _ established  procedures 

PPC’s  quick  decision  to 
secure  power  averted 

Hyd  leak  on  deck _ [Airframel  r  [simple  procedural  Evacuated  aircraft  potential  Injury _ 
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[Alfframel  r  [Simple  procedural  [Noflapldg 


Subject  Line _ Type  Class  Category _ Results _ CO  comments _ My  comments _ Judgment  Explain  Unfav 
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[Cracked  windshield _ |Airframe|  r  [Simple  procedural 


Subject  Line  _ Type  Class  Category _ Results _ CO  comments _ |My  comments _ |  Judgment  |  Explain  |  Unfav 
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Subject  Line  _ Type  Class  Category _ Results _ CO  comments _ My  comments _ Judgment  Explain  Unfav 

Interphdne  CB  -  |ost  Comm  Elec _ r  Simple  procedural _ _ , 

FOUO _ ^ _ Etec _ r  Simple  procedural _ ~ 

FQUO _ Elec _ r  Simple  procedural _ ; _ 

FOUO  Elec  r  Simple  procedural 
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Chips  light _ Eng _ r  Simple  procedural  3  eng  Idg 

Oil  leak _ Eng _ r  Simple  procedural  3  eng  Idg 

Chips  light _ Eng _ r  Simple  procedural  3  eng  idg 

Oil  leak _ Eng _ r  Simple  procedural  3  eng  Idg 
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Subject  Line  _ Type  Class  Category _ Results _ CO  comments _ My  comments _ Judgment  Explain  Unfav 

Fx  pum|)  failed  tp  terminate  Prop _ r  Simple  procedural  3  eng  Idg _  “ 

NTS  inop  light  Prop  r  Simple  procedural  3  eng  Idg 

NTS  inop  light  Prop  r  Simple  procedural  3  eng  Idg 
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Subject  Line  _ Type  Class  Category _ Resuits _ CO  comments _ My  comments _ Judgment  Explain  Unfav 

Binding 'pwr  lever  during  NTS  Prop _ r  Simple  procedural  4  eng  Idg _ 
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Unusual  Change  to  tech  man 

Autopilot  failed  to  disconnect  Airframe  r  malfunction _ Abort  mission _ submitted  for  a/p  preflight 

Unusual  ICS  jackbox  shocked  co- 

Elec  shock  Elec  r  malfunction  Abort  mission  pilot 
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APPENDIX  C.  SUMMARY  OF  RESULTS 


Appendix  C  contains  a  summary  table  of  the  results  presented  in  Chapter  IV.  The 
table  is  only  a  summary  of  the  three  tables  contained  in  Chapter  IV  and  has  no  new  or 
additional  information. 
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#  requiring  Total#  Total# 

Category _ Judgment  favorable _ Explanation  favorable _ unfavorable  Explanation  unfavorable 
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3  tT%  No  NATOPS  guidance 

Press  7  11% 

Prop  1  6%  _ _ _ 

ifl  _ Total  requiring  judgment _ <» 
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